Projects per year
Abstract
We introduce EPIC-SOUNDS, a large-scale dataset of audio annotations capturing temporal extents and class labels within the audio stream of the egocentric videos. We propose an annotation pipeline where annotators temporally label distinguishable audio segments and describe the action that could have caused this sound. We identify actions that can be discriminated purely from audio, through grouping these free-form descriptions of audio into classes. For actions that involve objects colliding, we collect human annotations of the materials of these objects (e.g. a glass object being placed on a wooden surface), which we verify from video, discarding ambiguities. Overall, EPIC-SOUNDS includes 78.4k categorised segments of audible events and actions, distributed across 44 classes as well as 39.2k non-categorised segments. We train and evaluate state-of-the-art audio recognition and detection models on our dataset, for both audio-only and audio-visual methods. We also conduct analysis on: the temporal overlap between audio events, the temporal
and label correlations between audio and visual modalities, the ambiguities in annotating materials from audio-only input, the importance of audio-only labels and the limitations of current models to understand actions that sound.
and label correlations between audio and visual modalities, the ambiguities in annotating materials from audio-only input, the importance of audio-only labels and the limitations of current models to understand actions that sound.
| Original language | English |
|---|---|
| Pages (from-to) | 9953-9965 |
| Number of pages | 13 |
| Journal | IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) |
| Volume | 47 |
| Issue number | 11 |
| Early online date | 18 Jul 2025 |
| DOIs | |
| Publication status | Published - 1 Nov 2025 |
Bibliographical note
Publisher Copyright:© 1979-2012 IEEE.
Fingerprint
Dive into the research topics of 'EPIC-SOUNDS: A Large-Scale Dataset of Actions that Sound'. Together they form a unique fingerprint.Projects
- 3 Finished
-
Visual AI - Full Programme Grant Extension
Damen, D. (Principal Investigator)
1/06/23 → 30/11/25
Project: Research
-
Visual AI - Full Programme Grant Extension
Damen, D. (Principal Investigator)
1/06/23 → 30/11/25
Project: Research
-
8459 EPSRC EP/T004991/1 UMPIRE - Dima Aldamen Fellowship
Damen, D. (Principal Investigator)
1/02/20 → 31/01/25
Project: Research
Student theses
-
Leveraging Multimodal Data for Egocentric Video Understanding
Chalk, J. (Author), 20 Jan 2026Student thesis: Doctoral Thesis › Doctor of Philosophy (PhD)
File
Datasets
-
EPIC-KITCHENS-100
Aldamen, D. (Creator), Kazakos, E. (Creator), Doughty, H. (Creator), Munro, J. (Creator), Price, W. (Creator), Wray, M. (Creator), Perrett, T. (Creator) & Ma, J. (Creator), University of Bristol, 15 May 2020
DOI: 10.5523/bris.2g1n6qdydwa9u22shpxqzp0t8m, http://data.bris.ac.uk/data/dataset/2g1n6qdydwa9u22shpxqzp0t8m
Dataset
Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver