Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness

Maell Cullen*, Ben Davey, Karl J. Friston, Rosalyn J. Moran

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

13 Citations (Scopus)
271 Downloads (Pure)

Abstract

Background: Artificial intelligence has recently attained humanlike performance in a number of gamelike domains. These advances have been spurred by brain-inspired architectures and algorithms such as hierarchical filtering and reinforcement learning. OpenAI Gym is an open-source platform in which to train, test, and benchmark algorithms—it provides a range of tasks, including those of classic arcade games such as Doom. Here we describe how the platform might be used as a simulation, test, and diagnostic paradigm for psychiatric conditions. Methods: To illustrate how active inference models of game play could be used to test mechanistic and algorithmic properties of psychiatric disorders, we provide two exemplar analyses. The first speaks to the impact of aging on cognition, examining game-play behaviors in a model of aging in which we compared age-dependent changes of younger (n = 9, 22 ± 1 years of age) and older (n = 7, 56 ± 5 years of age) adult players. The second is an illustration of a putative feature of anhedonia in which we simulated diminished sensitivity to reward. Results: These simulations demonstrate how active inference can be used to test predicted changes in both neurobiology and beliefs in psychiatric cohorts. We show that, as well as behavioral measures, putative neural correlates of active inference can be simulated, and hypothesized (model-based) differences in local field potentials and blood oxygen level–dependent responses can be produced. Conclusions: We show that active inference, through epistemic and value-based goals, enables simulated subjects to actively develop detailed representations of gaming environments, and we demonstrate the use of a principled algorithmic and neurobiological framework for testing hypotheses in psychiatric illness.

Original languageEnglish
Pages (from-to)809-818
Number of pages10
JournalBiological Psychiatry: Cognitive Neuroscience and Neuroimaging
Volume3
Issue number9
Early online date10 Jul 2018
DOIs
Publication statusPublished - Sep 2018

Keywords

  • Active inference
  • Computational phenotyping
  • Computational psychiatry
  • Free energy principle
  • Game-based imaging biomarkers
  • Markov decision process

Fingerprint Dive into the research topics of 'Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness'. Together they form a unique fingerprint.

Cite this