SEMBED: Semantic Embedding of Egocentric Action Videos

Michael Wray; Davide Moltisanti; Walterio Mayol-Cuevas; Dima Damen

doi:10.1007/978-3-319-46604-0_38

SEMBED: Semantic Embedding of Egocentric Action Videos

Michael Wray^*, Davide Moltisanti, Walterio Mayol-Cuevas, Dima Damen

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

7 Citations (Scopus)

365 Downloads (Pure)

Abstract

We present SEMBED, an approach for embedding an egocentric object interaction video in a semantic-visual graph to estimate the probability distribution over its potential semantic labels. When object interactions are annotated using unbounded choice of verbs, we embrace the wealth and ambiguity of these labels by capturing the semantic relationships as well as the visual similarities over motion and appearance features. We show how SEMBED can interpret a challenging dataset of
1225 freely annotated egocentric videos, outperforming SVM classification by more than 5%.

Original language	English
Title of host publication	Computer Vision – ECCV 2016 Workshops
Subtitle of host publication	Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part I
Editors	Gang Hua, Hervé Jégou
Publisher	Springer
Pages	532-545
Number of pages	14
ISBN (Electronic)	9783319466040
ISBN (Print)	9783319466033
DOIs	https://doi.org/10.1007/978-3-319-46604-0_38
Publication status	Published - 18 Sept 2016
Event	14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands Duration: 8 Oct 2016 → 16 Oct 2016

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	9913
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th European Conference on Computer Vision, ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	8/10/16 → 16/10/16

Keywords

Egocentric Action Recognition
Semantic Ambiguity
Semantic Embedding

Access to Document

10.1007/978-3-319-46604-0_38

Full-text PDF (accepted author manuscript)
This is the author accepted manuscript (AAM). The final published version (version of record) is available online via Springer at http://link.springer.com/chapter/10.1007%2F978-3-319-46604-0_38. Please refer to any applicable terms of use of the publisher.
Accepted author manuscript, 7.18 MB

Handle.net

Persistent link

Cite this

Wray, M., Moltisanti, D., Mayol-Cuevas, W., & Damen, D. (2016). SEMBED: Semantic Embedding of Egocentric Action Videos. In G. Hua, & H. Jégou (Eds.), Computer Vision – ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part I (pp. 532-545). (Lecture Notes in Computer Science; Vol. 9913). Springer. https://doi.org/10.1007/978-3-319-46604-0_38

@inproceedings{42bb079e2bb24d1dae5e6ed853c24eb7,

title = "SEMBED: Semantic Embedding of Egocentric Action Videos",

abstract = "We present SEMBED, an approach for embedding an egocentric object interaction video in a semantic-visual graph to estimate the probability distribution over its potential semantic labels. When object interactions are annotated using unbounded choice of verbs, we embrace the wealth and ambiguity of these labels by capturing the semantic relationships as well as the visual similarities over motion and appearance features. We show how SEMBED can interpret a challenging dataset of1225 freely annotated egocentric videos, outperforming SVM classification by more than 5%.",

keywords = "Egocentric Action Recognition, Semantic Ambiguity, Semantic Embedding",

author = "Michael Wray and Davide Moltisanti and Walterio Mayol-Cuevas and Dima Damen",

year = "2016",

month = sep,

day = "18",

doi = "10.1007/978-3-319-46604-0_38",

language = "English",

isbn = "9783319466033",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "532--545",

editor = "Gang Hua and Herv{\'e} J{\'e}gou",

booktitle = "Computer Vision – ECCV 2016 Workshops",

note = "14th European Conference on Computer Vision, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

}

Wray, M, Moltisanti, D, Mayol-Cuevas, W & Damen, D 2016, SEMBED: Semantic Embedding of Egocentric Action Videos. in G Hua & H Jégou (eds), Computer Vision – ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part I. Lecture Notes in Computer Science, vol. 9913, Springer, pp. 532-545, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 8/10/16. https://doi.org/10.1007/978-3-319-46604-0_38

SEMBED: Semantic Embedding of Egocentric Action Videos. / Wray, Michael; Moltisanti, Davide; Mayol-Cuevas, Walterio et al.
Computer Vision – ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part I. ed. / Gang Hua; Hervé Jégou. Springer, 2016. p. 532-545 (Lecture Notes in Computer Science; Vol. 9913).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - SEMBED

T2 - 14th European Conference on Computer Vision, ECCV 2016

AU - Wray, Michael

AU - Moltisanti, Davide

AU - Mayol-Cuevas, Walterio

AU - Damen, Dima

PY - 2016/9/18

Y1 - 2016/9/18

N2 - We present SEMBED, an approach for embedding an egocentric object interaction video in a semantic-visual graph to estimate the probability distribution over its potential semantic labels. When object interactions are annotated using unbounded choice of verbs, we embrace the wealth and ambiguity of these labels by capturing the semantic relationships as well as the visual similarities over motion and appearance features. We show how SEMBED can interpret a challenging dataset of1225 freely annotated egocentric videos, outperforming SVM classification by more than 5%.

AB - We present SEMBED, an approach for embedding an egocentric object interaction video in a semantic-visual graph to estimate the probability distribution over its potential semantic labels. When object interactions are annotated using unbounded choice of verbs, we embrace the wealth and ambiguity of these labels by capturing the semantic relationships as well as the visual similarities over motion and appearance features. We show how SEMBED can interpret a challenging dataset of1225 freely annotated egocentric videos, outperforming SVM classification by more than 5%.

KW - Egocentric Action Recognition

KW - Semantic Ambiguity

KW - Semantic Embedding

UR - http://www.scopus.com/inward/record.url?scp=84989953781&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46604-0_38

DO - 10.1007/978-3-319-46604-0_38

M3 - Conference Contribution (Conference Proceeding)

AN - SCOPUS:84989953781

SN - 9783319466033

T3 - Lecture Notes in Computer Science

SP - 532

EP - 545

BT - Computer Vision – ECCV 2016 Workshops

A2 - Hua, Gang

A2 - Jégou, Hervé

PB - Springer

Y2 - 8 October 2016 through 16 October 2016

ER -

SEMBED: Semantic Embedding of Egocentric Action Videos

Abstract

Publication series

Conference

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

GLANCE: GLANCE: GLAnceable Nuances for Contextual Events

Verbs and Me: An Investigation Into Verbs as Labels for Action Recognition in Video Understanding

Cite this

SEMBED: Semantic Embedding of Egocentric Action Videos

Abstract

Publication series

Conference

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

Projects

GLANCE: GLANCE: GLAnceable Nuances for Contextual Events

Student theses

Verbs and Me: An Investigation Into Verbs as Labels for Action Recognition in Video Understanding

Cite this