Perceptually-inspired super-resolution of compressed videos

Di Ma, Mariana Fernandez Afonso, Aaron Zhang, David Bull

Research output: Contribution to conferenceConference Paper

106 Downloads (Pure)

Abstract

Spatial resolution adaptation is a technique which has often been employed in video compression to enhance coding efficiency. This approach encodes a lower resolution version of the input video and reconstructs the original resolution during decoding. Instead of using conventional up-sampling filters, recent work has employed advanced super-resolution methods based on convolutional neural networks (CNNs) to further improve reconstruction quality. These approaches are usually trained to minimise pixel-based losses such as Mean-Squared Error (MSE), despite the fact that this type of loss metric does not correlate well with subjective opinions. In this paper, a perceptually-inspired
super-resolution approach (M-SRGAN) is proposed for spatial up-sampling of compressed video using a modified CNN model, which has been trained using a generative adversarial network (GAN) on compressed content with perceptual loss functions. The proposed method was integrated with HEVC HM 16.20, and has been evaluated on the JVET Common Test Conditions (UHD test sequences) using the Random Access configuration. The results show evident perceptual quality improvement over the original HM 16.20, with an average bitrate saving of 35.6% (Bjøntegaard Delta measurement) based on a perceptual quality metric, VMAF.
Original languageEnglish
Number of pages8
Publication statusPublished - 15 Aug 2019
EventSPIE Optics + Photonics: Connecting Minds, Advancing Light - San Diego, California, United States
Duration: 11 Aug 201915 Aug 2019
http://spie.org/conferences-and-exhibitions/optics-and-photonics

Conference

ConferenceSPIE Optics + Photonics
CountryUnited States
CitySan Diego, California
Period11/08/1915/08/19
Internet address

Keywords

  • HEVC
  • perceptual super-resolution
  • video compression
  • generative adversarial networks
  • Spatial resolution adaptation

Fingerprint Dive into the research topics of 'Perceptually-inspired super-resolution of compressed videos'. Together they form a unique fingerprint.

  • Cite this

    Ma, D., Fernandez Afonso, M., Zhang, A., & Bull, D. (2019). Perceptually-inspired super-resolution of compressed videos. Paper presented at SPIE Optics + Photonics, San Diego, California, United States.