A spin-glass model for the loss surfaces of generative adversarial networks

Nicholas P Baskerville*, Jonathan P Keating, Francesco Mezzadri, Joseph Najnudel

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

98 Downloads (Pure)

Abstract

We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting.
Original languageEnglish
Article number29
Number of pages45
JournalJournal of Statistical Physics
Volume186
Issue number2
DOIs
Publication statusPublished - 18 Jan 2022

Bibliographical note

Funding Information:
FM is grateful for support from the University Research Fellowship of the University of Bristol. JPK is pleased to acknowledge support from European Research Council Advanced Grant 740900 (LogCorRM). NPB is grateful to Diego Granziol for useful discussions (in particular suggesting Fig. ), to Jonathan Hodgson for help designing the contour plots and to the Advanced Computing Research Centre of the University of Bristol for the GPU resources to perform the experiments. The authors are grateful to several anonymous reviewers whose comments led to considerable improvements in this paper.

Publisher Copyright:
© 2022, Crown.

Keywords

  • math-ph
  • cond-mat.dis-nn
  • cond-mat.stat-mech
  • cs.LG
  • math.MP
  • math.PR

Fingerprint

Dive into the research topics of 'A spin-glass model for the loss surfaces of generative adversarial networks'. Together they form a unique fingerprint.
  • HPC (High Performance Computing) Facility

    Polly E Eccleston (Other), Simon H Atack (Other) & D A G Williams (Manager)

    Facility/equipment: Facility

Cite this