UnweaveNet: Unweaving Activity Stories

Will Price, Carl Vondrick, Dima Damen*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

34 Downloads (Pure)

Abstract

Our lives can be seen as a complex weaving of activi- ties; we switch from one activity to another, to maximise our achievements or in reaction to demands placed upon us. Observing a video of unscripted daily activities, we parse the video into its constituent activity threads through a process we call unweaving. To accomplish this, we introduce a video representation explicitly capturing activity threads called a thread bank, along with a neural controller capable of detecting goal changes and resuming of past activities, together forming UnweaveNet. We train and evaluate UnweaveNet on sequences from the unscripted egocentric dataset EPIC-KITCHENS. We propose and showcase the effi- cacy of pretraining UnweaveNet in a self-supervised manner.
Original languageEnglish
Title of host publication2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages13770-13779
Number of pages10
ISBN (Electronic)978-1-6654-6946-3
ISBN (Print)978-1-6654-6947-0
DOIs
Publication statusUnpublished - 24 Jun 2022
EventComputer Vision and Pattern Recognition (CVPR) - New Orleans, United States
Duration: 19 Jun 202225 Jun 2022

Publication series

NameProceedings. IEEE Conference on Computer Vision and Pattern Recognition
PublisherIEEE
ISSN (Print)1063-6919
ISSN (Electronic)2575-7075

Conference

ConferenceComputer Vision and Pattern Recognition (CVPR)
Country/TerritoryUnited States
Period19/06/2225/06/22

Fingerprint

Dive into the research topics of 'UnweaveNet: Unweaving Activity Stories'. Together they form a unique fingerprint.

Cite this