Dynamic reference frames in video coding

Abstract

The proliferation of high-resolution videos over the internet, including the introduction of 360 degree and 8K content, is driving the demand for compression technology of superior quality at lower bit rates. Video compression has been an active area of research over the last three decades, with new video compression standards such as H.266 / Versatile Video Coding continuously improving rate-quality performance. Apart from the progress made by compression using traditional techniques, recent years have seen the emergence of optimisation approaches based on machine learning, particularly deep neural networks. These new standards and techniques have not only improved compression performance, but also supported the delivery of more immersive viewer experiences over otherwise constrained IP networks.

This thesis explores innovative approaches to video compression, specifically targeting the development and selection of reference frames in the encoder. Reference frames are typically used as a basis for motion estimation and compensation, where the choice of reference frame(s) can significantly influence both the bitrate and the visual quality of the reconstructed content. New dynamic techniques are introduced in this thesis aimed at improving video codec gains. These techniques are applied to HEVC and VVC and provide significant gains.

The first technique explores spatiotemporal information from existing frames as Key Frames, exploiting redundancies to achieve compression improvements. The Key Frame search technique with JVET SDR test sequences achieves BD-RATE (Y) savings of 2.5% with the VVC codec. In addition, a new algorithm has been proposed to reduce the resources required and the complexity of the new Key Frame search by over 80%. The final technique presented in this thesis uses deep neural networks to exploit and optimise the reference frames using frame interpolation. The frame interpolation technique with JVET SDR test sequences exhibits BD-RATE (Y) savings of 2.3% with HEVC and 0.4% with VVC.

Date of Award	6 Dec 2022
Original language	English
Awarding Institution	University of Bristol
Sponsors	Nippon Telegraph & Telephone
Supervisor	Fan Zhang (Supervisor) & David R Bull (Supervisor)