Feature Blindness: A Challenge for Understanding and Modelling Visual Object Recognition

Gaurav Malhotra*, Marin Dujmovic, Jeffrey S Bowers

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

9 Citations (Scopus)
25 Downloads (Pure)

Abstract

Humans rely heavily on the shape of objects to recognise them. Recently, it has been argued that Convolutional Neural Networks (CNNs) can also show a shape-bias, provided their learning environment contains this bias. This has led to the proposal that CNNs provide good mechanistic models of shape-bias and, more generally, human visual processing. However, it is also possible that humans and CNNs show a shape-bias for very different reasons, namely, shape-bias in humans may be a consequence of architectural and cognitive constraints whereas CNNs show a shape-bias as a consequence of learning the statistics of the environment. We investigated this question by exploring shape-bias in humans and CNNs when they learn in a novel environment. We observed that, in this new environment, humans (i) focused on shape and overlooked many non-shape features, even when non-shape features were more diagnostic, (ii)
learned based on only one out of multiple predictive features, and (iii) failed to learn when global features, such as shape, were absent. This behaviour contrasted with the predictions of a statistical inference model with no priors, showing the strong role that shape-bias plays in human feature selection. It also contrasted with CNNs that (i) preferred to categorise objects based on non-shape features, and (ii) increased reliance on these non-shape features as they became more predictive. This was the case even when the CNN was pre-trained to have a shape-bias and the convolutional backbone was frozen. These results suggest that shape-bias has a different source in humans and CNNs: while learning in CNNs is driven by the statistical properties of the environment, humans are highly constrained by their previous biases, which suggests that cognitive constraints play a key role in how humans learn to recognise novel objects.
Original languageEnglish
Article numbere1009572
JournalPLoS Computational Biology
Volume18
Issue number5
DOIs
Publication statusPublished - 13 May 2022

Bibliographical note

Publisher Copyright:
Copyright: © 2022 Malhotra et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Fingerprint

Dive into the research topics of 'Feature Blindness: A Challenge for Understanding and Modelling Visual Object Recognition'. Together they form a unique fingerprint.

Cite this