ARDUOUS: Tutorial on Annotation of useR Data for UbiquitOUs Systems - Developing a Data Annotation Protocol

Emma L. Tonkin, Gregory J. L. Tourte, Teodor Stoev, Kristina Yordanova

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

5 Downloads (Pure)


Data annotation is key to a large number of fields, including ubiquitous computing. Documenting the quality and extent of annotation is increasingly recognised as an important aspect of understanding the validity, biases and limitations of systems built using this data: hence, it is also relevant to regulatory and compliance needs and outcomes. However, the process of annotation often receives little attention, and is characterised in the literature as “under-described” and “invisible work”. In this tutorial, we bring together existing resources and methods to present a framework for the iterative development and evaluation of an annotation protocol, from requirements gathering, setting scope, development, documentation, piloting and evaluation, through to scaling-up annotation processes for a production annotation process. We also explore the potential of semi-supervised approaches and state-of-the-art methods such as the use of generative AI in supporting annotation workflows, and how such approaches are validated and their strengths and weaknesses characterised. This tutorial is designed to be suitable for people from a wide range of backgrounds, as annotation can be understood as a highly interdisciplinary task and often requires collaboration with subject matter experts from relevant fields. Participants will trial and evaluate a selection of annotation interfaces and walk through the process of evaluating the outcomes. By the end of the workshop, participants will develop a deeper understanding of the task of developing an annotation protocol and aspects of the requirements and context which should be taken into account.

Presentations and code from this event will be shared openly on a Github repository.
Original languageEnglish
Title of host publicationUbicomp/ISWC '23 Adjunct
Subtitle of host publicationAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing
EditorsMonica Tentori, Nadir Weibel, Kristof Van Laerhoven, Zhongyi Zhou
Number of pages4
ISBN (Electronic) 9798400702006
Publication statusPublished - 8 Oct 2023
EventUbicomp/ ISWC 2023 - Cancun, Mexico
Duration: 8 Oct 202312 Oct 2023


ConferenceUbicomp/ ISWC 2023


  • data annotation
  • ubiquitous computing
  • activity recognition
  • methodology
  • framework


Dive into the research topics of 'ARDUOUS: Tutorial on Annotation of useR Data for UbiquitOUs Systems - Developing a Data Annotation Protocol'. Together they form a unique fingerprint.

Cite this