Skip to content

EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition

Research output: Contribution to conferencePaper

Original languageEnglish
Number of pages10
DateAccepted/In press - 22 Jul 2019
DatePublished (current) - 2 Nov 2019
EventIEEE/CVF International Conference on Computer Vision (ICCV) 2019 - Korea, Seoul
Duration: 27 Oct 20192 Nov 2019


ConferenceIEEE/CVF International Conference on Computer Vision (ICCV) 2019


We focus on multi-modal fusion for egocentric action recognition, and propose a novel architecture for multimodal temporal-binding, i.e. the combination of modalities within a range of temporal offsets. We train the architecture with three modalities – RGB, Flow and Audio – and combine them with mid-level fusion alongside sparse temporal sampling of fused representations. In contrast with previous works, modalities are fused before temporal aggregation, with shared modality and fusion weights over time. Our proposed architecture is trained end-to-end, outperforming individual modalities as well as late-fusion of modalities. We demonstrate the importance of audio in egocentric vision, on per-class basis, for identifying actions as well as interacting objects. Our method achieves state of the art results on both the seen and unseen test sets of the largest egocentric dataset: EPIC-Kitchens, on all metrics using the public leaderboard.


IEEE/CVF International Conference on Computer Vision (ICCV) 2019

Duration27 Oct 20192 Nov 2019
Location of eventKorea

Event: Conference

Download statistics

No data available



View research connections

Related faculties, schools or groups