Priorless Recurrent Networks Learn Curiously

Jeffrey J Mitchell, Jeffrey S Bowers

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

9 Citations (Scopus)
109 Downloads (Pure)

Abstract

Recently, domain-general recurrent neural networks, without explicit linguistic inductive biases, have been shown to successfully reproduce a range of human language behaviours, such as accurately predicting number agreement between nouns and verbs. We show that such networks will also learn number agreement within unnatural sentence structures, i.e. structures that are not found within any natural languages and which humans struggle to process. These results suggest that the models are learning from their input in a manner that is substantially different from human language acquisition, and we undertake an analysis of how the learned knowledge is stored in the weights of the network. We find that while the model has an effective understanding of singular versus plural for individual sentences, there is a lack of a unified concept of number agreement connecting these processes across the full range of inputs. Moreover, the weights handling natural and unnatural structures overlap substantially, in a way that underlines the non-human-like nature of the knowledge learned by the network.
Original languageEnglish
Title of host publicationProceedings of the 28th International Conference on Computational Linguistics
PublisherACL Anthology
Pages5147-5158
Number of pages12
ISBN (Print)978-1-952148-27-9
DOIs
Publication statusPublished - 1 Dec 2020
EventThe 28th International Conference on Computational Linguistics - Barcelona, Spain
Duration: 8 Dec 202013 Dec 2020
Conference number: 28
https://coling2020.org/

Conference

ConferenceThe 28th International Conference on Computational Linguistics
Abbreviated titleCOLING 2020
Country/TerritorySpain
CityBarcelona
Period8/12/2013/12/20
Internet address

Research Groups and Themes

  • Cognitive Science
  • Language

Fingerprint

Dive into the research topics of 'Priorless Recurrent Networks Learn Curiously'. Together they form a unique fingerprint.
  • M and M

    Bowers, J. S. (Principal Investigator)

    1/09/1731/08/22

    Project: Research, Parent

Cite this