Training neural networks to encode symbols enables combinatorial generalization

Ivan I. Vankov*, Jeffrey S. Bowers

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

2 Citations (Scopus)
54 Downloads (Pure)

Abstract

Combinatorial generalization-the ability to understand and produce novel combinations of already familiar elements-is considered to be a core capacity of the human mind and a major challenge to neural network models. A significant body of research suggests that conventional neural networks cannot solve this problem unless they are endowed with mechanisms specifically engineered for the purpose of representing symbols. In this paper, we introduce a novel way of representing symbolic structures in connectionist terms-the vectors approach to representing symbols (VARS), which allows training standard neural architectures to encode symbolic knowledge explicitly at their output layers. In two simulations, we show that neural networks not only can learn to produce VARS representations, but in doing so they achieve combinatorial generalization in their symbolic and non-symbolic output. This adds to other recent work that has shown improved combinatorial generalization under some training conditions, and raises the question of whether specific mechanisms or training routines are needed to support symbolic processing.

Original languageEnglish
Article number20190309
Number of pages10
JournalPhilosophical Transactions of the Royal Society B: Biological Sciences
Volume375
Issue number1791
DOIs
Publication statusPublished - 16 Dec 2019

Structured keywords

  • Language
  • Cognitive Science

Keywords

  • Combinatorial generalization
  • Neural networks
  • Symbols

Fingerprint Dive into the research topics of 'Training neural networks to encode symbols enables combinatorial generalization'. Together they form a unique fingerprint.

Cite this