Skip to content

Learning Visual Actions Using Multiple Verb-Only Labels

Research output: Contribution to conferencePaper

Original languageEnglish
Number of pages14
DateAccepted/In press - 1 Jul 2019
DatePublished (current) - 12 Sep 2019
Event30th British Machine Vision Conference - Cariff, United Kingdom
Duration: 9 Sep 201912 Sep 2019
Conference number: 30


Conference30th British Machine Vision Conference
Abbreviated titleBMVC
CountryUnited Kingdom
Internet address


This work introduces verb-only representations for both recognition and retrieval of visual actions, in video. Current methods neglect legitimate semantic ambiguities between verbs, instead choosing unambiguous subsets of verbs along with objects to disambiguate the actions. We instead propose multiple verb-only labels, which we learn through hard or soft assignment as a regression. This enables learning a much larger vocabulary of verbs, including contextual overlaps of these verbs. We collect multi-verb annotations for three action video datasets and evaluate the verb-only labelling representations for action recognition and cross-modal retrieval (video-to-text and text-to-video).
We demonstrate that multi-label verb-only representations outperform conventional single verb labels. We also explore other benefits of a multi-verb representation including cross-dataset retrieval and verb type (manner and result verb types) retrieval.


30th British Machine Vision Conference

Abbreviated titleBMVC
Conference number30
Duration9 Sep 201912 Sep 2019
CountryUnited Kingdom
Web address (URL)
Degree of recognitionNational event

Event: Conference



View research connections

Related faculties, schools or groups