Support for reduced presentation durations in subjective video quality assessment

Felix Mercer Moss, Chun-Ting Yeh, Fan Zhang, Roland Baddeley, David R. Bull

Research output: Contribution to journalArticle (Academic Journal)peer-review

7 Citations (Scopus)
319 Downloads (Pure)

Abstract

Video content distributors, codec developers and researchers in related fields often rely on subjective assessments to ensure that their video processing procedures result in satisfactory quality. The current 10 s recommendation for the length of test sequences in subjective video quality assessment, however, has recently been questioned. Not only do sequences of this length depart from modern cinematic shooting styles, the use of shorter sequences would also enable substantial efficiency improvements to the data collection process. Our previous work, using a double-stimulus methodology, indicated that shortening test sequences had a limited impact upon rating behaviour. Here, using a larger database and additional opinion score measures, we also explore the same effect within the popular single-stimulus approach. Two groups of viewers assessed reference and distorted videos ranging in length from 1.5 s to 10 s. Analyses confirmed our previous findings using the DSCQS paradigm, and were replicated when using a similar single-stimulus paradigm: while viewers' DMOS for 1.5 s videos was significantly lower than for 10 s, no significant variation was found between the groups of 10 s, 7 s and 5 s videos. Together with our previous research, these data lead us to recommend the use of 5 s, temporally-consistent video clips in quality assessment studies that employ either DSCQS or its single-stimulus variant. The extension of our recommendation to further methodologies is also discussed.
Original languageEnglish
Pages (from-to)38–49
Number of pages12
JournalSignal Processing: Image Communication
Volume48
Early online date17 Aug 2016
DOIs
Publication statusPublished - Oct 2016

Structured keywords

  • Cognitive Science
  • Visual Perception

Keywords

  • Subjective testing
  • Quality assessment
  • Test conditions
  • Reliability
  • Methodology
  • Video signal processing
  • Video sequences
  • Video databases
  • Visual perception
  • Statistical reliability
  • Double stimulus continuous quality scale
  • Single stimulus continuous quality scale
  • Mean opinion scores
  • Video coding
  • HEVC

Cite this