Exploiting Stereoscopic Disparity for Augmenting Human Activity Recognition Performance

Ioannis Pitas, Ioannis Mademlis, Alexandros Iosifidis, Anastasios Tefas, Nikos Nikolaidis

Research output: Contribution to journalArticle (Academic Journal)

274 Downloads (Pure)

Abstract

This work investigates several ways to exploit scene depth information, implicitly available through the modality of stereoscopic disparity in 3D videos, with the purpose of augmenting performance in the problem of recognizing complex human activities in natural settings. The standard state-of-the-art activity recognition algorithmic pipeline consists in the consecutive stages of video description, video representation and video classification. Multimodal, depth-aware modifications to standard methods are being proposed and studied, both for video description and for video representation, that indirectly incorporate scene geometry information derived from stereo disparity. At the description
level, this is made possible by suitably manipulating video interest points based on disparity data. At the representation level, the followed approach represents each video by multiple vectors corresponding to different disparity zones, resulting in multiple activity descriptions defined by disparity characteristics. In both cases, a scene segmentation is thus implicitly implemented, based on the distance of each imaged object from the camera during video acquisition. The investigated approaches are flexible and able to cooperate with any monocular low-level feature descriptor. They are evaluated using a publicly available activity recognition dataset of unconstrained stereoscopic 3D videos, consisting in
extracts from Hollywood movies, and compared both against competing depth-aware approaches and a state-of-the-art monocular algorithm. Quantitative evaluation reveals that some of the examined approaches achieve state-of-the-art performance.
Original languageEnglish
Pages (from-to)11641-11660
Number of pages20
JournalMultimedia Tools and Applications
Volume75
Issue number19
Early online date4 Jun 2015
DOIs
Publication statusPublished - Oct 2016

Keywords

  • Human Activity Recognition
  • Stereoscopic Video Description
  • Bag of Features
  • Disparity Zones

Fingerprint Dive into the research topics of 'Exploiting Stereoscopic Disparity for Augmenting Human Activity Recognition Performance'. Together they form a unique fingerprint.

  • Cite this