Abstract
We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting.
| Original language | English |
|---|---|
| Article number | 29 |
| Number of pages | 45 |
| Journal | Journal of Statistical Physics |
| Volume | 186 |
| Issue number | 2 |
| DOIs | |
| Publication status | Published - 18 Jan 2022 |
Bibliographical note
Funding Information:FM is grateful for support from the University Research Fellowship of the University of Bristol. JPK is pleased to acknowledge support from European Research Council Advanced Grant 740900 (LogCorRM). NPB is grateful to Diego Granziol for useful discussions (in particular suggesting Fig. ), to Jonathan Hodgson for help designing the contour plots and to the Advanced Computing Research Centre of the University of Bristol for the GPU resources to perform the experiments. The authors are grateful to several anonymous reviewers whose comments led to considerable improvements in this paper.
Publisher Copyright:
© 2022, Crown.
Keywords
- math-ph
- cond-mat.dis-nn
- cond-mat.stat-mech
- cs.LG
- math.MP
- math.PR
Fingerprint
Dive into the research topics of 'A spin-glass model for the loss surfaces of generative adversarial networks'. Together they form a unique fingerprint.Equipment
-
HPC (High Performance Computing) and HTC (High Throughput Computing) Facilities
Alam, S. R. (Manager), Williams, D. A. G. (Manager), Eccleston, P. E. (Manager) & Greene, D. (Manager)
Facility/equipment: Facility