THE MIND’S EYE: VISUALIZING CLASS-AGNOSTIC FEATURES OF CNNS

Alexandros Stergiou

doi:10.1109/ICIP42928.2021.9506287

THE MIND’S EYE: VISUALIZING CLASS-AGNOSTIC FEATURES OF CNNS

Alexandros Stergiou^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

1 Citation (Scopus)

Abstract

Visual interpretability of Convolutional Neural Networks (CNNs) has gained significant popularity because of the great challenges that CNN complexity imposes to understanding their inner workings. Although many techniques have been proposed to visualize class features of CNNs, most of them do not provide a correspondence between inputs and the extracted features in specific layers. This prevents the discovery of stimuli that each layer responds better to. We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Exploring features in this class-agnostic manner allows for a greater focus on the feature extractor of CNNs. Our method uses a dual-objective activation maximization and distance minimization loss, without requiring a generator network nor modifications to the original model. This limits the number of FLOPs to that of the original network. We demonstrate the visualization quality on widely-used architectures.¹

Original language	English
Title of host publication	2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings
Publisher	IEEE Computer Society
Pages	2738-2742
Number of pages	5
ISBN (Electronic)	9781665441155
DOIs	https://doi.org/10.1109/ICIP42928.2021.9506287
Publication status	Published - 23 Aug 2021
Event	2021 IEEE International Conference on Image Processing, ICIP 2021 - Anchorage, United States Duration: 19 Sept 2021 → 22 Sept 2021

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2021-September
ISSN (Print)	1522-4880
ISSN (Electronic)	2381-8549

Conference

Conference	2021 IEEE International Conference on Image Processing, ICIP 2021
Country/Territory	United States
City	Anchorage
Period	19/09/21 → 22/09/21

Bibliographical note

Funding Information:
Thanks to the Netherlands Organization for Scientific Research (NWO) for funding this research with TOP-C2 grant ARBITER. 1Code is available at https://git.io/JL9Wg and our demo video: https://youtu.be/Au3jaUdnPKM

Publisher Copyright:
© 2021 IEEE.

Keywords

CNN explainability
Convolutional features
Feature visualization

Access to Document

10.1109/ICIP42928.2021.9506287Licence: Unspecified

Handle.net

Persistent link

Cite this

@inproceedings{94ece3cdc13e48e8bf06e70f2d4f86a6,

title = "THE MIND{\textquoteright}S EYE: VISUALIZING CLASS-AGNOSTIC FEATURES OF CNNS",

abstract = "Visual interpretability of Convolutional Neural Networks (CNNs) has gained significant popularity because of the great challenges that CNN complexity imposes to understanding their inner workings. Although many techniques have been proposed to visualize class features of CNNs, most of them do not provide a correspondence between inputs and the extracted features in specific layers. This prevents the discovery of stimuli that each layer responds better to. We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Exploring features in this class-agnostic manner allows for a greater focus on the feature extractor of CNNs. Our method uses a dual-objective activation maximization and distance minimization loss, without requiring a generator network nor modifications to the original model. This limits the number of FLOPs to that of the original network. We demonstrate the visualization quality on widely-used architectures.1",

keywords = "CNN explainability, Convolutional features, Feature visualization",

author = "Alexandros Stergiou",

note = "Funding Information: Thanks to the Netherlands Organization for Scientific Research (NWO) for funding this research with TOP-C2 grant ARBITER. 1Code is available at https://git.io/JL9Wg and our demo video: https://youtu.be/Au3jaUdnPKM Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE International Conference on Image Processing, ICIP 2021 ; Conference date: 19-09-2021 Through 22-09-2021",

year = "2021",

month = aug,

day = "23",

doi = "10.1109/ICIP42928.2021.9506287",

language = "English",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE Computer Society",

pages = "2738--2742",

booktitle = "2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings",

address = "United States",

}

Stergiou, A 2021, THE MIND’S EYE: VISUALIZING CLASS-AGNOSTIC FEATURES OF CNNS. in 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. Proceedings - International Conference on Image Processing, ICIP, vol. 2021-September, IEEE Computer Society, pp. 2738-2742, 2021 IEEE International Conference on Image Processing, ICIP 2021, Anchorage, United States, 19/09/21. https://doi.org/10.1109/ICIP42928.2021.9506287

THE MIND’S EYE: VISUALIZING CLASS-AGNOSTIC FEATURES OF CNNS. / Stergiou, Alexandros.
2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. IEEE Computer Society, 2021. p. 2738-2742 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2021-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - THE MIND’S EYE

T2 - 2021 IEEE International Conference on Image Processing, ICIP 2021

AU - Stergiou, Alexandros

N1 - Funding Information: Thanks to the Netherlands Organization for Scientific Research (NWO) for funding this research with TOP-C2 grant ARBITER. 1Code is available at https://git.io/JL9Wg and our demo video: https://youtu.be/Au3jaUdnPKM Publisher Copyright: © 2021 IEEE.

PY - 2021/8/23

Y1 - 2021/8/23

N2 - Visual interpretability of Convolutional Neural Networks (CNNs) has gained significant popularity because of the great challenges that CNN complexity imposes to understanding their inner workings. Although many techniques have been proposed to visualize class features of CNNs, most of them do not provide a correspondence between inputs and the extracted features in specific layers. This prevents the discovery of stimuli that each layer responds better to. We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Exploring features in this class-agnostic manner allows for a greater focus on the feature extractor of CNNs. Our method uses a dual-objective activation maximization and distance minimization loss, without requiring a generator network nor modifications to the original model. This limits the number of FLOPs to that of the original network. We demonstrate the visualization quality on widely-used architectures.1

AB - Visual interpretability of Convolutional Neural Networks (CNNs) has gained significant popularity because of the great challenges that CNN complexity imposes to understanding their inner workings. Although many techniques have been proposed to visualize class features of CNNs, most of them do not provide a correspondence between inputs and the extracted features in specific layers. This prevents the discovery of stimuli that each layer responds better to. We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Exploring features in this class-agnostic manner allows for a greater focus on the feature extractor of CNNs. Our method uses a dual-objective activation maximization and distance minimization loss, without requiring a generator network nor modifications to the original model. This limits the number of FLOPs to that of the original network. We demonstrate the visualization quality on widely-used architectures.1

KW - CNN explainability

KW - Convolutional features

KW - Feature visualization

UR - https://www.scopus.com/pages/publications/85117779390

U2 - 10.1109/ICIP42928.2021.9506287

DO - 10.1109/ICIP42928.2021.9506287

M3 - Conference Contribution (Conference Proceeding)

AN - SCOPUS:85117779390

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 2738

EP - 2742

BT - 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings

PB - IEEE Computer Society

Y2 - 19 September 2021 through 22 September 2021

ER -

THE MIND’S EYE: VISUALIZING CLASS-AGNOSTIC FEATURES OF CNNS

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this