Explaining Activities as Consistent Groups of Events: A Bayesian Framework using Attribute Multiset Grammars

Dima Damen, David Hogg

Research output: Contribution to journalArticle (Academic Journal)peer-review

8 Citations (Scopus)


We propose a method for disambiguating uncertain detections of events by seeking global explanations for activities. Given a noisy visual input, and exploiting our knowledge of the activity and its constraints, one can provide a consistent set of events explaining all the detections. The paper presents a complete framework that starts with a general way to formalise the set of global explanations for a given activity using attribute multiset grammars (AMG). An AMG combines the event hierarchy with the necessary features for recognition and algebraic constraints defining allowable combinations of events and features. Parsing a set of detections by such a grammar finds a consistent set of events that satisfies the activity’s constraints. Each parse tree has a posterior probability in a Bayesian sense. To find the best parse tree, the grammar and a finite set of detections are mapped into a Bayesian network. The set of possible labellings of the Bayesian network corresponds to the set of all parse trees for a given set of detections. We compare greedy, multiple-hypotheses trees, reversible jump MCMC, and integer programming for finding the Maximum a Posteriori (MAP) solution over the space of explanations. The framework is tested for two applications; the activity in a bicycle rack and around a building entrance.
Original languageEnglish
Pages (from-to)83-102
Number of pages14
JournalInternational Journal of Computer Vision
Issue number1
Publication statusPublished - May 2012


  • Activity Analysis
  • Event Recognition
  • Global Explanation


Dive into the research topics of 'Explaining Activities as Consistent Groups of Events: A Bayesian Framework using Attribute Multiset Grammars'. Together they form a unique fingerprint.

Cite this