UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment

Xinyi Wang; Angeliki Katsenou; David R Bull

doi:10.1117/12.2676136

UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment

Xinyi Wang^*, Angeliki Katsenou, David R Bull

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

26 Downloads (Pure)

Abstract

The volume of User Generated Content (UGC) has increased in recent years. The challenge with this type of content is assessing its quality. So far, the state-of-the-art metrics are not exhibiting a very high correlation with perceptual quality. In this paper, we explore state-of-the-art metrics that extract/combine natural scene statistics and deep neural network features. We experiment with these by introducing saliency maps to improve perceptibility. We train and test our models using public datasets, namely, YouTube-UGC and KoNViD-1k. Preliminary results indicate that high correlations are achieved by using only deep features while adding saliency is not always boosting the performance. Our results and code will be made publicly available to serve as a benchmark for the research community and can be found on our project page: https://github.com/xinyiW915/SPIE-2023-Supplementary.

Original language	English
Title of host publication	Applications of Digital Image Processing XLVI
Editors	Andrew G. Tescher, Touradj Ebrahimi
Publisher	Society of Photo-Optical Instrumentation Engineers (SPIE)
Number of pages	15
Volume	12674
ISBN (Electronic)	9781510665620
DOIs	https://doi.org/10.1117/12.2676136
Publication status	Published - 24 Aug 2023
Event	SPIE Optical Engineering + Applications: Applications of Digital Image Processing XLVI - San Diego, California, United States , San Diego, United States Duration: 20 Aug 2023 → 24 Aug 2023 https://spie.org/OP

Publication series

Name	Proceedings of SPIE - The International Society for Optical Engineering
Volume	12674
ISSN (Print)	0277-786X
ISSN (Electronic)	1996-756X

Conference

Conference	SPIE Optical Engineering + Applications
Country/Territory	United States
City	San Diego
Period	20/08/23 → 24/08/23
Internet address	https://spie.org/OP

Bibliographical note

Publisher Copyright:
© 2023 SPIE.

Keywords

cs.CV
cs.MM

Access to Document

10.1117/12.2676136

Full-text PDF (accepted author manuscript)
This is the accepted author manuscript (AAM). The final published version (version of record) is available online via SPIE at https://doi.org/10.1117/12.2676136. Please refer to any applicable terms of use of the publisher.
Accepted author manuscript, 21.9 MB

Handle.net

Persistent link

HPC (High Performance Computing) and HTC (High Throughput Computing) Facilities
Sadaf R Alam (Manager), Steven A Chapman (Manager), Polly E Eccleston (Other), Simon H Atack (Other) & D A G Williams (Manager)
Facility/equipment: Facility

Cite this

Wang, X., Katsenou, A., & Bull, D. R. (2023). UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment. In A. G. Tescher, & T. Ebrahimi (Eds.), Applications of Digital Image Processing XLVI (Vol. 12674). Article 1267418 (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 12674). Society of Photo-Optical Instrumentation Engineers (SPIE). https://doi.org/10.1117/12.2676136

Wang, Xinyi ; Katsenou, Angeliki ; Bull, David R. / UGC Quality Assessment : Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment. Applications of Digital Image Processing XLVI. editor / Andrew G. Tescher ; Touradj Ebrahimi. Vol. 12674 Society of Photo-Optical Instrumentation Engineers (SPIE), 2023. (Proceedings of SPIE - The International Society for Optical Engineering).

@inproceedings{fc58db11f9324a01812cf967018f29da,

title = "UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment",

abstract = "The volume of User Generated Content (UGC) has increased in recent years. The challenge with this type of content is assessing its quality. So far, the state-of-the-art metrics are not exhibiting a very high correlation with perceptual quality. In this paper, we explore state-of-the-art metrics that extract/combine natural scene statistics and deep neural network features. We experiment with these by introducing saliency maps to improve perceptibility. We train and test our models using public datasets, namely, YouTube-UGC and KoNViD-1k. Preliminary results indicate that high correlations are achieved by using only deep features while adding saliency is not always boosting the performance. Our results and code will be made publicly available to serve as a benchmark for the research community and can be found on our project page: https://github.com/xinyiW915/SPIE-2023-Supplementary.",

keywords = "cs.CV, cs.MM",

author = "Xinyi Wang and Angeliki Katsenou and Bull, {David R}",

note = "Publisher Copyright: {\textcopyright} 2023 SPIE.; SPIE Optical Engineering + Applications : Applications of Digital Image Processing XLVI ; Conference date: 20-08-2023 Through 24-08-2023",

year = "2023",

month = aug,

day = "24",

doi = "10.1117/12.2676136",

language = "English",

volume = "12674",

series = "Proceedings of SPIE - The International Society for Optical Engineering",

publisher = "Society of Photo-Optical Instrumentation Engineers (SPIE)",

editor = "Tescher, {Andrew G.} and Touradj Ebrahimi",

booktitle = "Applications of Digital Image Processing XLVI",

address = "United States",

url = "https://spie.org/OP",

}

Wang, X , Katsenou, A & Bull, DR 2023, UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment. in AG Tescher & T Ebrahimi (eds), Applications of Digital Image Processing XLVI. vol. 12674, 1267418 , Proceedings of SPIE - The International Society for Optical Engineering, vol. 12674, Society of Photo-Optical Instrumentation Engineers (SPIE), SPIE Optical Engineering + Applications, San Diego, United States, 20/08/23. https://doi.org/10.1117/12.2676136

UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment. / Wang, Xinyi ; Katsenou, Angeliki ; Bull, David R.
Applications of Digital Image Processing XLVI. ed. / Andrew G. Tescher; Touradj Ebrahimi. Vol. 12674 Society of Photo-Optical Instrumentation Engineers (SPIE), 2023. 1267418 (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 12674).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - UGC Quality Assessment

T2 - SPIE Optical Engineering + Applications

AU - Wang, Xinyi

AU - Katsenou, Angeliki

AU - Bull, David R

PY - 2023/8/24

Y1 - 2023/8/24

N2 - The volume of User Generated Content (UGC) has increased in recent years. The challenge with this type of content is assessing its quality. So far, the state-of-the-art metrics are not exhibiting a very high correlation with perceptual quality. In this paper, we explore state-of-the-art metrics that extract/combine natural scene statistics and deep neural network features. We experiment with these by introducing saliency maps to improve perceptibility. We train and test our models using public datasets, namely, YouTube-UGC and KoNViD-1k. Preliminary results indicate that high correlations are achieved by using only deep features while adding saliency is not always boosting the performance. Our results and code will be made publicly available to serve as a benchmark for the research community and can be found on our project page: https://github.com/xinyiW915/SPIE-2023-Supplementary.

AB - The volume of User Generated Content (UGC) has increased in recent years. The challenge with this type of content is assessing its quality. So far, the state-of-the-art metrics are not exhibiting a very high correlation with perceptual quality. In this paper, we explore state-of-the-art metrics that extract/combine natural scene statistics and deep neural network features. We experiment with these by introducing saliency maps to improve perceptibility. We train and test our models using public datasets, namely, YouTube-UGC and KoNViD-1k. Preliminary results indicate that high correlations are achieved by using only deep features while adding saliency is not always boosting the performance. Our results and code will be made publicly available to serve as a benchmark for the research community and can be found on our project page: https://github.com/xinyiW915/SPIE-2023-Supplementary.

KW - cs.CV

KW - cs.MM

U2 - 10.1117/12.2676136

DO - 10.1117/12.2676136

M3 - Conference Contribution (Conference Proceeding)

VL - 12674

T3 - Proceedings of SPIE - The International Society for Optical Engineering

BT - Applications of Digital Image Processing XLVI

A2 - Tescher, Andrew G.

A2 - Ebrahimi, Touradj

PB - Society of Photo-Optical Instrumentation Engineers (SPIE)

Y2 - 20 August 2023 through 24 August 2023

ER -

Wang X , Katsenou A , Bull DR. UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment. In Tescher AG, Ebrahimi T, editors, Applications of Digital Image Processing XLVI. Vol. 12674. Society of Photo-Optical Instrumentation Engineers (SPIE). 2023. 1267418 . (Proceedings of SPIE - The International Society for Optical Engineering). doi: 10.1117/12.2676136

UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Handle.net

Fingerprint

Equipment

HPC (High Performance Computing) and HTC (High Throughput Computing) Facilities

Cite this