Ego4D: Around the World in 3,000 Hours of Egocentric Video

Kristen Grauman*, Michael Wray, Adriano Fragomeni, Jonathan P N Munro, Will Price, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Bernard Ghanem, C.V. Jawahar, Kris Kitani, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Mike Zheng Shou, Antonio Torrallba, Jitendra Malik

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

306 Downloads (Pure)

Abstract

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of daily- life activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards, with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cam- eras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception. Project Page:
https://ego4d-data.org/
Original languageEnglish
Title of host publication2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages18973-18990
Number of pages18
ISBN (Electronic)978-1-6654-6946-3
ISBN (Print)978-1-6654-6947-0
DOIs
Publication statusPublished - 27 Sept 2022
EventComputer Vision and Pattern Recognition (CVPR) - New Orleans, United States
Duration: 19 Jun 202225 Jun 2022

Publication series

NameProceedings. IEEE Conference on Computer Vision and Pattern Recognition
PublisherIEEE
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

ConferenceComputer Vision and Pattern Recognition (CVPR)
Country/TerritoryUnited States
Period19/06/2225/06/22

Fingerprint

Dive into the research topics of 'Ego4D: Around the World in 3,000 Hours of Egocentric Video'. Together they form a unique fingerprint.

Cite this