When the Ground Truth is not True: Modelling Human Biases in Temporal Annotations

In supervised learning, low quality annotations lead to poorly performing classification and detection models, while also rendering evaluation unreliable. Annotation quality is affected by multiple factors. For example, in the post-hoc self-reporting of daily activities, cognitive biases are one of the most common ingredients.
In particular, reporting the start and duration of an activity after its finalisation may incorporate biases introduced by personal time perceptions, as well as the imprecision and lack of granularity affected by time rounding. When dealing with time-bounded data, the annotations' consistency over the event is particularly important for both event detection and classification. Here we propose a method to model human biases on temporal annotations and proposed the use of soft labels. Experimental results in synthetic data show that soft labels are a better approximation of the ground truth for several metrics. We showcase the method on a real dataset of daily activities.