You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video

Dima Damen (Aldamen), Teesid Leelasawassuk, Osian Haines, Andrew Calway, Walterio Mayol-Cuevas

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

50 Citations (Scopus)

Abstract

We present a fully unsupervised approach for i) discovery of task-relevant objects and ii) how these objects have been used. Given egocentric video from multiple operators, the approach can discover objects with which the users interact, both static objects such as a coffee machine as well as movable ones such as a cup. Importantly, the common modes of interaction for discovered objects are also found. We investigate using appearance, position, motion and attention, and present results using each and a combination of relevant features. Results show that the method is capable of discovering 95% of task-relevant objects on a variety of daily tasks such as initialising a printer, preparing a coffee or setting up a gym machine. In addition, the approach enables the automatic generation of guidance video on how these objects have been used before.
Original languageEnglish
Title of host publicationProceedings of the British Machine Vision Conference
PublisherBritish Machine Vision Association (BMVC Proceedings)
ISBN (Electronic)1901725529
Publication statusPublished - 2014
EventBritish Machine Vision Conference - Surrey, United Kingdom
Duration: 4 Sep 20127 Sep 2012

Conference

ConferenceBritish Machine Vision Conference
Country/TerritoryUnited Kingdom
CitySurrey
Period4/09/127/09/12

Structured keywords

  • Digital Health
  • SPHERE

Fingerprint

Dive into the research topics of 'You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video'. Together they form a unique fingerprint.

Cite this