Abstract
Automatic shot selection is an important aspect of movie summarization that is helpful both to producers and to audiences, e.g., for market promotion or browsing purposes. However, most of the related research has focused on shot
selection based on low-level video content, which disregards semantic information, or on narrative properties extracted from text, which requires the movie script to be available. In this work, semantic shot selection based on the narrative prominence of movie characters in both the visual and the audio modalities is investigated, without the need for additional data such as a script.
The output is a movie summary that only contains video frames from selected movie shots. Selection is controlled by a userprovided shot retention parameter, that removes key-frames/keysegments from the skim based on actor face appearances and speech instances. This novel process (Multimodal Shot Pruning,
or MSP) is algebraically modelled as a multimodal matrix Column Subset Selection Problem, which is solved using an evolutionary computing approach.
selection based on low-level video content, which disregards semantic information, or on narrative properties extracted from text, which requires the movie script to be available. In this work, semantic shot selection based on the narrative prominence of movie characters in both the visual and the audio modalities is investigated, without the need for additional data such as a script.
The output is a movie summary that only contains video frames from selected movie shots. Selection is controlled by a userprovided shot retention parameter, that removes key-frames/keysegments from the skim based on actor face appearances and speech instances. This novel process (Multimodal Shot Pruning,
or MSP) is algebraically modelled as a multimodal matrix Column Subset Selection Problem, which is solved using an evolutionary computing approach.
Original language | English |
---|---|
Title of host publication | 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016) |
Subtitle of host publication | Proceedings of a meeting held 21-23 September 2016, Montreal, Quebec, Canada |
Publisher | Institute of Electrical and Electronics Engineers (IEEE) |
Pages | 352-356 |
Number of pages | 5 |
ISBN (Electronic) | 9781509037247 |
ISBN (Print) | 9781509037254 |
DOIs | |
Publication status | Published - Feb 2017 |
Event | 18th International Workshop on Multimedia Signal Processing: MMSP2016 - Montreal, Canada Duration: 21 Sep 2016 → 23 Sep 2016 |
Publication series
Name | Proceedings of the International Workshop on Multimedia Signal Processing (MMSP) |
---|---|
Publisher | Institute of Electrical and Electronics Engineers (IEEE) |
ISSN (Print) | 2163-3517 |
ISSN (Electronic) | 2473-3628 |
Conference
Conference | 18th International Workshop on Multimedia Signal Processing |
---|---|
Country | Canada |
City | Montreal |
Period | 21/09/16 → 23/09/16 |
Keywords
- Video Summarization
- Shot Selection
- Column Sub-set Selection Problem