Movie shot selection preserving narrative properties

Ioannis Mademlis; Anastasios Tefas; Nikos Nikolaidis; Ioannis Pitas

doi:10.1109/MMSP.2016.7813397

Movie shot selection preserving narrative properties

Ioannis Mademlis, Anastasios Tefas, Nikos Nikolaidis, Ioannis Pitas

Department of Electrical & Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

4 Citations (Scopus)

237 Downloads (Pure)

Abstract

Automatic shot selection is an important aspect of movie summarization that is helpful both to producers and to audiences, e.g., for market promotion or browsing purposes. However, most of the related research has focused on shot
selection based on low-level video content, which disregards semantic information, or on narrative properties extracted from text, which requires the movie script to be available. In this work, semantic shot selection based on the narrative prominence of movie characters in both the visual and the audio modalities is investigated, without the need for additional data such as a script.
The output is a movie summary that only contains video frames from selected movie shots. Selection is controlled by a userprovided shot retention parameter, that removes key-frames/keysegments from the skim based on actor face appearances and speech instances. This novel process (Multimodal Shot Pruning,
or MSP) is algebraically modelled as a multimodal matrix Column Subset Selection Problem, which is solved using an evolutionary computing approach.

Original language	English
Title of host publication	2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016)
Subtitle of host publication	Proceedings of a meeting held 21-23 September 2016, Montreal, Quebec, Canada
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	352-356
Number of pages	5
ISBN (Electronic)	9781509037247
ISBN (Print)	9781509037254
DOIs	https://doi.org/10.1109/MMSP.2016.7813397
Publication status	Published - Feb 2017
Event	18th International Workshop on Multimedia Signal Processing: MMSP2016 - Montreal, Canada Duration: 21 Sep 2016 → 23 Sep 2016

Publication series

Name	Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
ISSN (Print)	2163-3517
ISSN (Electronic)	2473-3628

Conference

Conference	18th International Workshop on Multimedia Signal Processing
Country	Canada
City	Montreal
Period	21/09/16 → 23/09/16

Keywords

Video Summarization
Shot Selection
Column Sub-set Selection Problem

Access to Document

10.1109/MMSP.2016.7813397

Full-text PDF (accepted author manuscript)
This is the accepted author manuscript (AAM). The final published version (version of record) is available online via IEEE at http://ieeexplore.ieee.org/document/7813397/. Please refer to any applicable terms of use of the publisher.
Accepted author manuscript, 182 KB

Fingerprint Dive into the research topics of 'Movie shot selection preserving narrative properties'. Together they form a unique fingerprint.

Cite this

Mademlis, I., Tefas, A., Nikolaidis, N., & Pitas, I. (2017). Movie shot selection preserving narrative properties. In 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016): Proceedings of a meeting held 21-23 September 2016, Montreal, Quebec, Canada (pp. 352-356). (Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/MMSP.2016.7813397

Mademlis, Ioannis ; Tefas, Anastasios ; Nikolaidis, Nikos ; Pitas, Ioannis. / Movie shot selection preserving narrative properties. 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016): Proceedings of a meeting held 21-23 September 2016, Montreal, Quebec, Canada. Institute of Electrical and Electronics Engineers (IEEE), 2017. pp. 352-356 (Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)).

@inproceedings{ecf62778937849a897a8132f33615e60,

title = "Movie shot selection preserving narrative properties",

abstract = "Automatic shot selection is an important aspect of movie summarization that is helpful both to producers and to audiences, e.g., for market promotion or browsing purposes. However, most of the related research has focused on shotselection based on low-level video content, which disregards semantic information, or on narrative properties extracted from text, which requires the movie script to be available. In this work, semantic shot selection based on the narrative prominence of movie characters in both the visual and the audio modalities is investigated, without the need for additional data such as a script.The output is a movie summary that only contains video frames from selected movie shots. Selection is controlled by a userprovided shot retention parameter, that removes key-frames/keysegments from the skim based on actor face appearances and speech instances. This novel process (Multimodal Shot Pruning,or MSP) is algebraically modelled as a multimodal matrix Column Subset Selection Problem, which is solved using an evolutionary computing approach.",

keywords = "Video Summarization, Shot Selection, Column Sub-set Selection Problem",

author = "Ioannis Mademlis and Anastasios Tefas and Nikos Nikolaidis and Ioannis Pitas",

year = "2017",

month = feb,

doi = "10.1109/MMSP.2016.7813397",

language = "English",

isbn = "9781509037254",

series = "Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "352--356",

booktitle = "2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016)",

address = "United States",

note = " 18th International Workshop on Multimedia Signal Processing : MMSP2016 ; Conference date: 21-09-2016 Through 23-09-2016",

}

Mademlis, I, Tefas, A, Nikolaidis, N & Pitas, I 2017, Movie shot selection preserving narrative properties. in 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016): Proceedings of a meeting held 21-23 September 2016, Montreal, Quebec, Canada. Proceedings of the International Workshop on Multimedia Signal Processing (MMSP), Institute of Electrical and Electronics Engineers (IEEE), pp. 352-356, 18th International Workshop on Multimedia Signal Processing, Montreal, Canada, 21/09/16. https://doi.org/10.1109/MMSP.2016.7813397

Movie shot selection preserving narrative properties. / Mademlis, Ioannis; Tefas, Anastasios; Nikolaidis, Nikos; Pitas, Ioannis.

2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016): Proceedings of a meeting held 21-23 September 2016, Montreal, Quebec, Canada. Institute of Electrical and Electronics Engineers (IEEE), 2017. p. 352-356 (Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Movie shot selection preserving narrative properties

AU - Mademlis, Ioannis

AU - Tefas, Anastasios

AU - Nikolaidis, Nikos

AU - Pitas, Ioannis

PY - 2017/2

Y1 - 2017/2

N2 - Automatic shot selection is an important aspect of movie summarization that is helpful both to producers and to audiences, e.g., for market promotion or browsing purposes. However, most of the related research has focused on shotselection based on low-level video content, which disregards semantic information, or on narrative properties extracted from text, which requires the movie script to be available. In this work, semantic shot selection based on the narrative prominence of movie characters in both the visual and the audio modalities is investigated, without the need for additional data such as a script.The output is a movie summary that only contains video frames from selected movie shots. Selection is controlled by a userprovided shot retention parameter, that removes key-frames/keysegments from the skim based on actor face appearances and speech instances. This novel process (Multimodal Shot Pruning,or MSP) is algebraically modelled as a multimodal matrix Column Subset Selection Problem, which is solved using an evolutionary computing approach.

AB - Automatic shot selection is an important aspect of movie summarization that is helpful both to producers and to audiences, e.g., for market promotion or browsing purposes. However, most of the related research has focused on shotselection based on low-level video content, which disregards semantic information, or on narrative properties extracted from text, which requires the movie script to be available. In this work, semantic shot selection based on the narrative prominence of movie characters in both the visual and the audio modalities is investigated, without the need for additional data such as a script.The output is a movie summary that only contains video frames from selected movie shots. Selection is controlled by a userprovided shot retention parameter, that removes key-frames/keysegments from the skim based on actor face appearances and speech instances. This novel process (Multimodal Shot Pruning,or MSP) is algebraically modelled as a multimodal matrix Column Subset Selection Problem, which is solved using an evolutionary computing approach.

KW - Video Summarization

KW - Shot Selection

KW - Column Sub-set Selection Problem

U2 - 10.1109/MMSP.2016.7813397

DO - 10.1109/MMSP.2016.7813397

M3 - Conference Contribution (Conference Proceeding)

SN - 9781509037254

T3 - Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)

SP - 352

EP - 356

BT - 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016)

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 18th International Workshop on Multimedia Signal Processing

Y2 - 21 September 2016 through 23 September 2016

ER -

Mademlis I, Tefas A, Nikolaidis N, Pitas I. Movie shot selection preserving narrative properties. In 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP 2016): Proceedings of a meeting held 21-23 September 2016, Montreal, Quebec, Canada. Institute of Electrical and Electronics Engineers (IEEE). 2017. p. 352-356. (Proceedings of the International Workshop on Multimedia Signal Processing (MMSP)). https://doi.org/10.1109/MMSP.2016.7813397