Visual voice activity detection based on spatiotemporal information and bag of words

Foteini Patrona; Alexandros Iosifidis; Anastasios Tefas; Nikos Nikolaidis; Ioannis Pitas

doi:10.1109/ICIP.2015.7351219

Visual voice activity detection based on spatiotemporal information and bag of words

Foteini Patrona, Alexandros Iosifidis, Anastasios Tefas, Nikos Nikolaidis, Ioannis Pitas

Department of Electrical & Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

2 Citations (Scopus)

383 Downloads (Pure)

Abstract

A novel method for Visual Voice Activity Detection (V-VAD) that exploits local shape and motion information appearing at spatiotemporal locations of interest for facial region video description and the Bag of Words (BoW) model for facial
region video representation is proposed in this paper. Facial region video classification is subsequently performed based on Single-hidden Layer Feedforward Neural (SLFN) network trained by applying the recently proposed kernel Extreme Learning Machine (kELM) algorithm on training facial videos depicting talking and non-talking persons. Experimental results on two publicly available V-VAD data sets, denote the effectiveness of the proposed method, since better generalization performance in unseen users is achieved, compared to recently proposed state-of-the-art methods.

Original language	English
Title of host publication	2015 IEEE International Conference on Image Processing (ICIP 2015)
Subtitle of host publication	Proceedings of a meeting held 27-30 September 2015, Quebec City, Quebec, Canada
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	2334-2338
Number of pages	5
ISBN (Electronic)	9781479983391
ISBN (Print)	9781479983407
DOIs	https://doi.org/10.1109/ICIP.2015.7351219
Publication status	Published - Jan 2016
Event	2015 IEEE International Conference on Image Processing (ICIP) - Quebec City, ON, Canada Duration: 27 Sept 2015 → 30 Sept 2015

Conference

Conference	2015 IEEE International Conference on Image Processing (ICIP)
Country/Territory	Canada
City	Quebec City, ON
Period	27/09/15 → 30/09/15

Keywords

Voice Activity Detection
Space-Time Interest Points
Bag of Words model
kernel Extreme Learning Machine

Access to Document

10.1109/ICIP.2015.7351219

Ioannis Pitas - Visual voice activity detection based on spatiotemporal information and bag of words - 2015Accepted author manuscript, 81.4 KB

Handle.net

Persistent link

Cite this

Patrona, F., Iosifidis, A., Tefas, A., Nikolaidis, N., & Pitas, I. (2016). Visual voice activity detection based on spatiotemporal information and bag of words. In 2015 IEEE International Conference on Image Processing (ICIP 2015): Proceedings of a meeting held 27-30 September 2015, Quebec City, Quebec, Canada (pp. 2334-2338). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICIP.2015.7351219

Patrona, Foteini ; Iosifidis, Alexandros ; Tefas, Anastasios et al. / Visual voice activity detection based on spatiotemporal information and bag of words. 2015 IEEE International Conference on Image Processing (ICIP 2015): Proceedings of a meeting held 27-30 September 2015, Quebec City, Quebec, Canada. Institute of Electrical and Electronics Engineers (IEEE), 2016. pp. 2334-2338

@inproceedings{a49b82a0e40044319304e4ac7dcae225,

title = "Visual voice activity detection based on spatiotemporal information and bag of words",

abstract = "A novel method for Visual Voice Activity Detection (V-VAD) that exploits local shape and motion information appearing at spatiotemporal locations of interest for facial region video description and the Bag of Words (BoW) model for facialregion video representation is proposed in this paper. Facial region video classification is subsequently performed based on Single-hidden Layer Feedforward Neural (SLFN) network trained by applying the recently proposed kernel Extreme Learning Machine (kELM) algorithm on training facial videos depicting talking and non-talking persons. Experimental results on two publicly available V-VAD data sets, denote the effectiveness of the proposed method, since better generalization performance in unseen users is achieved, compared to recently proposed state-of-the-art methods.",

keywords = "Voice Activity Detection, Space-Time Interest Points, Bag of Words model, kernel Extreme Learning Machine",

author = "Foteini Patrona and Alexandros Iosifidis and Anastasios Tefas and Nikos Nikolaidis and Ioannis Pitas",

year = "2016",

month = jan,

doi = "10.1109/ICIP.2015.7351219",

language = "English",

isbn = "9781479983407",

pages = "2334--2338",

booktitle = "2015 IEEE International Conference on Image Processing (ICIP 2015)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

note = "2015 IEEE International Conference on Image Processing (ICIP) ; Conference date: 27-09-2015 Through 30-09-2015",

}

Patrona, F, Iosifidis, A, Tefas, A, Nikolaidis, N & Pitas, I 2016, Visual voice activity detection based on spatiotemporal information and bag of words. in 2015 IEEE International Conference on Image Processing (ICIP 2015): Proceedings of a meeting held 27-30 September 2015, Quebec City, Quebec, Canada. Institute of Electrical and Electronics Engineers (IEEE), pp. 2334-2338, 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, ON, Canada, 27/09/15. https://doi.org/10.1109/ICIP.2015.7351219

Visual voice activity detection based on spatiotemporal information and bag of words. / Patrona, Foteini; Iosifidis, Alexandros; Tefas, Anastasios et al.
2015 IEEE International Conference on Image Processing (ICIP 2015): Proceedings of a meeting held 27-30 September 2015, Quebec City, Quebec, Canada. Institute of Electrical and Electronics Engineers (IEEE), 2016. p. 2334-2338.

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Visual voice activity detection based on spatiotemporal information and bag of words

AU - Patrona, Foteini

AU - Iosifidis, Alexandros

AU - Tefas, Anastasios

AU - Nikolaidis, Nikos

AU - Pitas, Ioannis

PY - 2016/1

Y1 - 2016/1

N2 - A novel method for Visual Voice Activity Detection (V-VAD) that exploits local shape and motion information appearing at spatiotemporal locations of interest for facial region video description and the Bag of Words (BoW) model for facialregion video representation is proposed in this paper. Facial region video classification is subsequently performed based on Single-hidden Layer Feedforward Neural (SLFN) network trained by applying the recently proposed kernel Extreme Learning Machine (kELM) algorithm on training facial videos depicting talking and non-talking persons. Experimental results on two publicly available V-VAD data sets, denote the effectiveness of the proposed method, since better generalization performance in unseen users is achieved, compared to recently proposed state-of-the-art methods.

AB - A novel method for Visual Voice Activity Detection (V-VAD) that exploits local shape and motion information appearing at spatiotemporal locations of interest for facial region video description and the Bag of Words (BoW) model for facialregion video representation is proposed in this paper. Facial region video classification is subsequently performed based on Single-hidden Layer Feedforward Neural (SLFN) network trained by applying the recently proposed kernel Extreme Learning Machine (kELM) algorithm on training facial videos depicting talking and non-talking persons. Experimental results on two publicly available V-VAD data sets, denote the effectiveness of the proposed method, since better generalization performance in unseen users is achieved, compared to recently proposed state-of-the-art methods.

KW - Voice Activity Detection

KW - Space-Time Interest Points

KW - Bag of Words model

KW - kernel Extreme Learning Machine

U2 - 10.1109/ICIP.2015.7351219

DO - 10.1109/ICIP.2015.7351219

M3 - Conference Contribution (Conference Proceeding)

SN - 9781479983407

SP - 2334

EP - 2338

BT - 2015 IEEE International Conference on Image Processing (ICIP 2015)

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 2015 IEEE International Conference on Image Processing (ICIP)

Y2 - 27 September 2015 through 30 September 2015

ER -

Patrona F, Iosifidis A, Tefas A, Nikolaidis N, Pitas I. Visual voice activity detection based on spatiotemporal information and bag of words. In 2015 IEEE International Conference on Image Processing (ICIP 2015): Proceedings of a meeting held 27-30 September 2015, Quebec City, Quebec, Canada. Institute of Electrical and Electronics Engineers (IEEE). 2016. p. 2334-2338 Epub 2015 Sept 27. doi: 10.1109/ICIP.2015.7351219

Visual voice activity detection based on spatiotemporal information and bag of words

Abstract

Conference

Keywords

Access to Document

Handle.net

Fingerprint

Cite this