Saliency-based deep convolutional neural network for no-reference image quality assessment

Sen Jia*, Yang Zhang

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

60 Citations (Scopus)
430 Downloads (Pure)

Abstract

In this paper, we proposed a novel method for No-Reference Image Quality Assessment (NR-IQA) by combining deep Convolutional Neural Network (CNN) with saliency map. We first investigate the effect of depth of CNNs for NR-IQA by comparing our proposed ten-layer Deep CNN (DCNN) for NR-IQA with the state-of-the-art CNN architecture proposed by Kang et al. (2014). Our results show that the DCNN architecture can deliver a higher accuracy on the LIVE dataset. To mimic human vision, we introduce saliency maps combining with CNN to propose a Saliency-based DCNN (SDCNN) framework for NR-IQA. We compute a saliency map for each image and both the map and the image are split into small patches. Each image patch is assigned with a patch importance value based on its saliency patch. A set of Salient Image Patches (SIPs) are selected according to their saliency and we only apply the model on those SIPs to predict the quality score for the whole image. Our experimental results show that the SDCNN framework is superior to other state-of-the-art approaches on the widely used LIVE dataset. The TID2008 and the CISQ image quality datasets are utilised to report cross-dataset results. The results indicate that our proposed SDCNN can generalise well on other datasets.

Original languageEnglish
Pages (from-to)1-14
Number of pages14
JournalMultimedia Tools and Applications
Early online date22 Aug 2017
DOIs
Publication statusPublished - 2018

Keywords

  • CNN
  • NR-IQA
  • Saliency map

Fingerprint

Dive into the research topics of 'Saliency-based deep convolutional neural network for no-reference image quality assessment'. Together they form a unique fingerprint.

Cite this