Simulation-Based Inference with Modern Generative Modelling

Abstract

In this work we discuss various methodologies for inferring the parameters of a model when the likelihood is intractable and is only known via a simulator, so-called simulation-based inference (SBI). In recent years there have been promising developments in the field of SBI resulting from the adoption of machine learning techniques. For example, generative modelling, a related yet distinct topic to likelihood-free inference, is a fast moving and very active area of research, whose ideas have already proved very successful in likelihood-free inference. In lieu of this fact, in this work we propose novel algorithms for likelihood-free inference tasks inspired by work from the generative modelling literature.

Motivated by the recent success of SBI methods which leverage density ratio estimation, we propose Variational Likelihood-Free Gradient Descent wherein a density ratio function is estimated which allows for the transportation of particles to the target distribution. Although novel in its approach, ultimately we find that it’s performance is subpar for a number of reasons which are elaborated on.

Recently a group of methods referred to as score-based diffusion models has proved state-of-the-art at a wide range of generative modelling tasks. The model learned in this framework allows for the transportation of samples from a tractable Gaussian distribution to the target distribution. To achieve this, the gradient of the logarithm of the target density is learned. The training procedure for the model is highly stable, straightforward to implement, and requires little tuning. Typically the model is a deep neural network. A crucial property of this paradigm, unlike that of the competitor methods, is that no architectural restrictions are imparted on our model. This fact enables us to specify highly flexible models which in turn can capture highly complex relationships.

We propose two algorithms in this framework: one which directly targets the posterior, Sequential Neural Posterior Score Estimation, and one which directly targets the likelihood, Sequential Neural Likelihood Score Estimation. These algorithms bridge score-based diffusion models to the SBI literature. The proposed algorithms give promising results across a wide range of experiments.

Date of Award	4 Feb 2025
Original language	English
Awarding Institution	University of Bristol
Supervisor	Song Liu (Supervisor) & Mark A Beaumont (Supervisor)