A multicue Bayesian state estimator for gaze prediction in open signed video

SJC Davies; D Agrafiotis; CN Canagarajah; DR Bull

doi:10.1109/TMM.2008.2008916

A multicue Bayesian state estimator for gaze prediction in open signed video

SJC Davies, D Agrafiotis, CN Canagarajah, DR Bull

Research output: Contribution to journal › Article (Academic Journal) › peer-review

5 Citations (Scopus)

426 Downloads (Pure)

Abstract

We propose a multicue gaze prediction framework for open signed video content, the benefits of which include coding gains without loss of perceived quality. We investigate which cues are relevant for gaze prediction and find that shot changes, facial orientation of the signer and face locations are the most useful. We then design a face orientation tracker based upon grid-based likelihood ratio trackers, using profile and frontal face detections. These cues are combined using a grid-based Bayesian state estimation algorithm to form a probability surface for each frame. We find that this gaze predictor outperforms a static gaze prediction and one based on face locations within the frame.

Translated title of the contribution	A multicue Bayesian state estimator for gaze prediction in open signed video
Original language	English
Pages (from-to)	39 - 48
Number of pages	10
Journal	IEEE Transactions on Multimedia
Volume	11
Issue number	1
DOIs	https://doi.org/10.1109/TMM.2008.2008916
Publication status	Published - Jan 2009

Bibliographical note

Publisher: IEEE
Rose publication type: Journal article

Sponsorship: The work of SJC Davies was supported
by the British Broadcasting Corporation (BBC).

Terms of use: Copyright © 2009 IEEE. Reprinted from IEEE Transactions on Multimedia.

This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Bristol's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org.

By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

Keywords

eye-tracking
face detection
gaze prediction
video coding

Access to Document

10.1109/TMM.2008.2008916

Davies IEEE Trans Multimedia Jan2009Accepted author manuscript, 1.42 MB

http://hdl.handle.net/1983/1394

Cite this

@article{8b4c9f9d96e74c029ad9152c2beeaaa7,

title = "A multicue Bayesian state estimator for gaze prediction in open signed video",

abstract = "We propose a multicue gaze prediction framework for open signed video content, the benefits of which include coding gains without loss of perceived quality. We investigate which cues are relevant for gaze prediction and find that shot changes, facial orientation of the signer and face locations are the most useful. We then design a face orientation tracker based upon grid-based likelihood ratio trackers, using profile and frontal face detections. These cues are combined using a grid-based Bayesian state estimation algorithm to form a probability surface for each frame. We find that this gaze predictor outperforms a static gaze prediction and one based on face locations within the frame.",

keywords = "eye-tracking, face detection, gaze prediction, video coding",

author = "SJC Davies and D Agrafiotis and CN Canagarajah and DR Bull",

note = "Publisher: IEEE Rose publication type: Journal article Sponsorship: The work of SJC Davies was supported by the British Broadcasting Corporation (BBC). Terms of use: Copyright {\textcopyright} 2009 IEEE. Reprinted from IEEE Transactions on Multimedia. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Bristol's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it. ",

year = "2009",

month = jan,

doi = "10.1109/TMM.2008.2008916",

language = "English",

volume = "11",

pages = "39 -- 48",

journal = "IEEE Transactions on Multimedia",

issn = "1520-9210",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

number = "1",

}

TY - JOUR

T1 - A multicue Bayesian state estimator for gaze prediction in open signed video

AU - Davies, SJC

AU - Agrafiotis, D

AU - Canagarajah, CN

AU - Bull, DR

N1 - Publisher: IEEE Rose publication type: Journal article Sponsorship: The work of SJC Davies was supported by the British Broadcasting Corporation (BBC). Terms of use: Copyright © 2009 IEEE. Reprinted from IEEE Transactions on Multimedia. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Bristol's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

PY - 2009/1

Y1 - 2009/1

N2 - We propose a multicue gaze prediction framework for open signed video content, the benefits of which include coding gains without loss of perceived quality. We investigate which cues are relevant for gaze prediction and find that shot changes, facial orientation of the signer and face locations are the most useful. We then design a face orientation tracker based upon grid-based likelihood ratio trackers, using profile and frontal face detections. These cues are combined using a grid-based Bayesian state estimation algorithm to form a probability surface for each frame. We find that this gaze predictor outperforms a static gaze prediction and one based on face locations within the frame.

AB - We propose a multicue gaze prediction framework for open signed video content, the benefits of which include coding gains without loss of perceived quality. We investigate which cues are relevant for gaze prediction and find that shot changes, facial orientation of the signer and face locations are the most useful. We then design a face orientation tracker based upon grid-based likelihood ratio trackers, using profile and frontal face detections. These cues are combined using a grid-based Bayesian state estimation algorithm to form a probability surface for each frame. We find that this gaze predictor outperforms a static gaze prediction and one based on face locations within the frame.

KW - eye-tracking

KW - face detection

KW - gaze prediction

KW - video coding

UR - http://www.scopus.com/inward/record.url?eid=2-s2.0-72549104842&partnerID=8YFLogxK

U2 - 10.1109/TMM.2008.2008916

DO - 10.1109/TMM.2008.2008916

M3 - Article (Academic Journal)

VL - 11

SP - 39

EP - 48

JO - IEEE Transactions on Multimedia

JF - IEEE Transactions on Multimedia

SN - 1520-9210

IS - 1

ER -

A multicue Bayesian state estimator for gaze prediction in open signed video

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this