On the Optimal Presentation Duration for Subjective Video Quality Assessment

Subjective quality assessment is an essential component of modern image and video processing, both for the validation of objective metrics and for the comparison of coding methods. However, the standard procedures used to collect data can be prohibitively time-consuming. One way of increasing the efficiency of data collection is to reduce the duration of test sequences from the 10 second length currently used in most subjective video quality assessment experiments. Here, we explore the impact of reducing sequence length upon perceptual accuracy when identifying compression artefacts. A group of four reference sequences, together with five levels of distortion, are used to compare the subjective ratings of viewers watching videos between 1.5 and 10 seconds long. We identify a smooth function indicating that accuracy increases linearly as the length of the sequences increases from 1.5 seconds to 7 seconds. The accuracy of observers viewing 1.5 second sequences was significantly inferior to those viewing sequences of 5 seconds, 7 seconds and 10 seconds. We argue that sequences between 5 seconds and 10 seconds produce satisfactory levels of accuracy but the practical benefits of acquiring more data lead us to recommend the use of 5 second sequences for future video quality assessment studies that use the DSCQS methodology.