Learning to Learn Sequential Dynamics

: Context-Aware Out-of-Distribution Adaptation for Time Series and Physical Systems

Abstract

Sequence models are at the forefront of the deep learning revolution, with far-reaching applications in the physical sciences and engineering. A critical challenge, however, is adapting these models to novel settings or contexts, a problem formally known as out-of-distribution (OoD) generalisation. While many algorithms leveraging the meta-learning ("learning to learn") paradigm show great promise, significant gaps remain in their explainability, effectiveness, and computational efficiency.

This thesis advances the development of efficient models capable of few- and zero-shot OoD adaptation on sequences and other modalities. The first part provides foundational background and a holistic literature review, identifying several key research questions that are addressed in the second part through the development of novel context-aware meta-learning frameworks. The first core contribution introduces a fast Taylor-based adaptation algorithm with improved interpretability and uncertainty estimation. The algorithm is subsequently extended to tackle problems beyond the forecasting of time-dependent physical systems. The third contribution introduces a new problem of generalisation across system hierarchies, demonstrating both the effectiveness and limitations of mixtures of experts as potential foundation models. A final contribution reinterprets meta-learning as automatic adaptation via linear recurrences in weight space, unlocking in-context learning and the ability to embed continuous physical priors within discrete sequence models.

Collectively, these results point towards exciting future directions that pave the way for artificial general intelligence through models that can thoroughly understand the physical world beyond their training data, and adapt correspondingly.

Date of Award	9 Dec 2025
Original language	English
Awarding Institution	University of Bristol
Supervisor	Tom Deakin (Supervisor), David A W Barton (Supervisor) & Simon N McIntosh-Smith (Supervisor)