Subtractive gating improves generalization in working memory tasks

Research output: Contribution to conferenceConference Posterpeer-review

12 Downloads (Pure)

Abstract

It is largely unclear how the brain learns to generalize to new situations. Although deep learning models offer great promise as potential models of the brain, they break down when tested on novel conditions not present in their training datasets. One of the most successful models in machine learning are gated-recurrent neural networks. Because of its working memory properties here we refer to these networks as working memory networks (WMN). We compare WMNs with a biologically motivated variant of these networks. In contrast to the multiplicative gating used by WMNs, this new variant operates via subtracting gating (subWMN). We tested these two models in a range of working memory tasks: orientation recall with distractors, orientation recall with addition distractors and memory addition, and a more challenging task: sequence recognition based on the machine learning handwritten digits dataset. We evaluated the generalization properties of these two networks for working memory tasks by measuring how well they copped with three working memory loads: memory maintenance over time, making memories distractor-resistant and memory updating. Across these tests subWMNs perform better and more robustly than WMNs. Overall, our work suggests that the brain may rely on subtractive gating for improved generalization in working memory tasks.
Original languageEnglish
Number of pages4
DOIs
Publication statusPublished - 14 Sep 2019
Event2019 Conference on Cognitive Computational Neuroscience - Berlin, Germany, Berlin, Germany
Duration: 13 Sep 201916 Sep 2019
https://ccneuro.org/2019/

Conference

Conference2019 Conference on Cognitive Computational Neuroscience
Abbreviated titleccneuro
CountryGermany
CityBerlin
Period13/09/1916/09/19
Internet address

Keywords

  • Gating
  • Working memory
  • Recurrent neural networks
  • Cortical Circuits

Fingerprint Dive into the research topics of 'Subtractive gating improves generalization in working memory tasks'. Together they form a unique fingerprint.

Cite this