Hiding a plane with a pixel: examining shape-bias in CNNs and the benefit of building in biological constraints

Research output: Contribution to journalArticle (Academic Journal)peer-review

39 Downloads (Pure)

Abstract

When deep convolutional neural networks (CNNs) are trained “end-to-end” on raw data, some of the feature detectors they develop in their early layers resemble the representations found in early visual cortex. This result has been used to draw parallels between deep learning systems and human visual perception. In this study, we show that when CNNs are trained end-to-end they learn to classify images based on whatever feature is predictive of a category within the dataset. This can lead to bizarre results where CNNs learn idiosyncratic features such as high-frequency noise-like masks. In the extreme case, our results demonstrate image categorisation on the basis of a single pixel. Such features are extremely unlikely to play any role in human object recognition, where experiments have repeatedly shown a strong preference for shape. Through a series of empirical studies with standard high-performance CNNs, we show that these networks do not develop a shape-bias merely through regularisation methods or more ecologically plausible training regimes. These results raise doubts over the assumption that simply learning end-to-end in standard CNNs leads to the emergence of similar representations to the human visual system. In the second part of the paper, we show that CNNs are less reliant on these idiosyncratic features when we forgo end-to-end learning and introduce hard-wired Gabor filters designed to mimic early visual processing in V1.
Original languageEnglish
Pages (from-to)57-68
JournalVision Research
Volume174
Early online date28 Jun 2020
DOIs
Publication statusE-pub ahead of print - 28 Jun 2020

Structured keywords

  • Visual Perception
  • Cognitive Science

Keywords

  • convolutional neural networks
  • shape-bias
  • image classification
  • object recognition
  • Gabor filters
  • end-to-end learning
  • biological constraints
  • V1

Fingerprint Dive into the research topics of 'Hiding a plane with a pixel: examining shape-bias in CNNs and the benefit of building in biological constraints'. Together they form a unique fingerprint.

Cite this