You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video

Dima Damen (Aldamen); Teesid Leelasawassuk; Osian Haines; Andrew Calway; Walterio Mayol-Cuevas

You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video

Dima Damen (Aldamen)
, Teesid Leelasawassuk
, Osian Haines
, Andrew Calway
, Walterio Mayol-Cuevas

Visual Information Laboratory

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

115 Citations (Scopus)

Abstract

We present a fully unsupervised approach for i) discovery of task-relevant objects and ii) how these objects have been used. Given egocentric video from multiple operators, the approach can discover objects with which the users interact, both static objects such as a coffee machine as well as movable ones such as a cup. Importantly, the common modes of interaction for discovered objects are also found. We investigate using appearance, position, motion and attention, and present results using each and a combination of relevant features. Results show that the method is capable of discovering 95% of task-relevant objects on a variety of daily tasks such as initialising a printer, preparing a coffee or setting up a gym machine. In addition, the approach enables the automatic generation of guidance video on how these objects have been used before.

Original language	English
Title of host publication	Proceedings of the British Machine Vision Conference
Publisher	British Machine Vision Association (BMVC Proceedings)
ISBN (Electronic)	1901725529
Publication status	Published - 2014
Event	British Machine Vision Conference - Surrey, United Kingdom Duration: 4 Sept 2012 → 7 Sept 2012

Conference

Conference	British Machine Vision Conference
Country/Territory	United Kingdom
City	Surrey
Period	4/09/12 → 7/09/12

Research Groups and Themes

Digital Health
SPHERE

Access to Document

http://www.bmva.org/bmvc/2014/papers/paper059/index.html

Handle.net

Persistent link

SPHERE (EPSRC IRC)
Craddock, I. J. (Principal Investigator), Coyle, D. T. (Principal Investigator), Flach, P. A. (Principal Investigator), Kaleshi, D. (Principal Investigator), Mirmehdi, M. (Principal Investigator), Piechocki, R. J. (Principal Investigator), Stark, B. H. (Principal Investigator), Ascione, R. (Co-Principal Investigator), Ashburn, A. M. (Collaborator), Burnett, M. E. (Collaborator), Damen, D. (Co-Principal Investigator), Gooberman-Hill, R. (Principal Investigator), Harwin, W. S. (Collaborator), Hilton, G. (Co-Principal Investigator), Holderbaum, W. (Collaborator), Holley, A. P. (Manager), Manchester, V. A. (Administrator), Meller, B. J. (Other ), Stack, E. (Collaborator) & Gilchrist, I. D. (Principal Investigator)
1/10/13 → 30/09/18
Project: Research, Parent

Cite this

Damen (Aldamen), D., Leelasawassuk, T., Haines, O., Calway, A., & Mayol-Cuevas, W. (2014). You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video. In Proceedings of the British Machine Vision Conference British Machine Vision Association (BMVC Proceedings). http://www.bmva.org/bmvc/2014/papers/paper059/index.html

@inproceedings{22a14bb5a4c14b398d1347773a7f0b48,

title = "You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video",

abstract = "We present a fully unsupervised approach for i) discovery of task-relevant objects and ii) how these objects have been used. Given egocentric video from multiple operators, the approach can discover objects with which the users interact, both static objects such as a coffee machine as well as movable ones such as a cup. Importantly, the common modes of interaction for discovered objects are also found. We investigate using appearance, position, motion and attention, and present results using each and a combination of relevant features. Results show that the method is capable of discovering 95\% of task-relevant objects on a variety of daily tasks such as initialising a printer, preparing a coffee or setting up a gym machine. In addition, the approach enables the automatic generation of guidance video on how these objects have been used before.",

author = "\{Damen (Aldamen)\}, Dima and Teesid Leelasawassuk and Osian Haines and Andrew Calway and Walterio Mayol-Cuevas",

year = "2014",

language = "English",

booktitle = "Proceedings of the British Machine Vision Conference",

publisher = "British Machine Vision Association (BMVC Proceedings)",

note = "British Machine Vision Conference ; Conference date: 04-09-2012 Through 07-09-2012",

}

Damen (Aldamen), D, Leelasawassuk, T, Haines, O, Calway, A & Mayol-Cuevas, W 2014, You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video. in Proceedings of the British Machine Vision Conference. British Machine Vision Association (BMVC Proceedings), British Machine Vision Conference, Surrey, United Kingdom, 4/09/12. <http://www.bmva.org/bmvc/2014/papers/paper059/index.html>

You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video. / Damen (Aldamen), Dima; Leelasawassuk, Teesid; Haines, Osian et al.
Proceedings of the British Machine Vision Conference. British Machine Vision Association (BMVC Proceedings), 2014.

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video

AU - Damen (Aldamen), Dima

AU - Leelasawassuk, Teesid

AU - Haines, Osian

AU - Calway, Andrew

AU - Mayol-Cuevas, Walterio

PY - 2014

Y1 - 2014

N2 - We present a fully unsupervised approach for i) discovery of task-relevant objects and ii) how these objects have been used. Given egocentric video from multiple operators, the approach can discover objects with which the users interact, both static objects such as a coffee machine as well as movable ones such as a cup. Importantly, the common modes of interaction for discovered objects are also found. We investigate using appearance, position, motion and attention, and present results using each and a combination of relevant features. Results show that the method is capable of discovering 95% of task-relevant objects on a variety of daily tasks such as initialising a printer, preparing a coffee or setting up a gym machine. In addition, the approach enables the automatic generation of guidance video on how these objects have been used before.

AB - We present a fully unsupervised approach for i) discovery of task-relevant objects and ii) how these objects have been used. Given egocentric video from multiple operators, the approach can discover objects with which the users interact, both static objects such as a coffee machine as well as movable ones such as a cup. Importantly, the common modes of interaction for discovered objects are also found. We investigate using appearance, position, motion and attention, and present results using each and a combination of relevant features. Results show that the method is capable of discovering 95% of task-relevant objects on a variety of daily tasks such as initialising a printer, preparing a coffee or setting up a gym machine. In addition, the approach enables the automatic generation of guidance video on how these objects have been used before.

M3 - Conference Contribution (Conference Proceeding)

BT - Proceedings of the British Machine Vision Conference

PB - British Machine Vision Association (BMVC Proceedings)

T2 - British Machine Vision Conference

Y2 - 4 September 2012 through 7 September 2012

ER -

You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video

Abstract

Conference

Research Groups and Themes

Access to Document

Handle.net

Fingerprint

Projects

SPHERE (EPSRC IRC)

Cite this