Projects per year
Abstract
Combinatorial generalisation — the ability to understand and produce novel combinations of familiar elements — is a core capacity of human intelligence that current AI systems struggle with. Recently, it has been suggested that learning disentangled representations may help address this problem. It is claimed that such representations should be able to capture the compositional structure of the world which can then be combined to support combinatorial generalisation. In this study, we systematically tested how the degree of disentanglement affects various forms of generalisation, including two forms of combinatorial generalisation that varied in difficulty. We trained three classes of variational autoencoders (VAEs) on two datasets on an unsupervised task by excluding combinations of generative factors during training. At test time we ask the models to reconstruct the missing combinations in order to measure generalisation performance. Irrespective of the degree of disentanglement, we found that the models supported only weak combinatorial generalisation. We obtained the same outcome when we directly input perfectly disentangled representations as the latents, and when we tested a model on a more complex task that explicitly required independent generative factors to be controlled. While learning disentangled representations does improve interpretability and sample efficiency in some downstream tasks, our results suggest that they are not sufficient for supporting more difficult forms of generalisation.
One-sentence Summary: Disentangled models do not achieve compositional generalization when tested systematically.
One-sentence Summary: Disentangled models do not achieve compositional generalization when tested systematically.
Original language | English |
---|---|
Title of host publication | International Conference on Learning Representations |
Publisher | OpenReview |
Publication status | Published - 12 Jun 2021 |
Event | ICLR 2025: The Thirteenth International Conference on Learning Representations - Singapore EXPO, Singapore, Singapore Duration: 24 Apr 2025 → 28 Apr 2025 https://iclr.cc/Conferences/2025 |
Conference
Conference | ICLR 2025 |
---|---|
Country/Territory | Singapore |
City | Singapore |
Period | 24/04/25 → 28/04/25 |
Internet address |
Fingerprint
Dive into the research topics of 'The role of disentanglement in generalisation'. Together they form a unique fingerprint.Projects
- 1 Finished