Robust neural posterior estimation and statistical model criticism

Computer simulations have proven a valuable tool for understanding complex
phenomena across the sciences. However, the utility of simulators for modelling
and forecasting purposes is often restricted by low data quality, as well as practical
limits to model fidelity. In order to circumvent these difficulties, we argue that
modellers must treat simulators as idealistic representations of the true data
generating process, and consequently should thoughtfully consider the risk of
model misspecification. In this work we revisit neural posterior estimation (NPE),
a class of algorithms that enable black-box parameter inference in simulation
models, and consider the implication of a simulation-to-reality gap. While recent
works have demonstrated reliable performance of these methods, the analyses
have been performed using synthetic data generated by the simulator model itself,
and have therefore only addressed the well-specified case. In this paper, we find
that the presence of misspecification, in contrast, leads to unreliable inference
when NPE is used naïvely. As a remedy we argue that principled scientific inquiry
with simulators should incorporate a model criticism component, to facilitate
interpretable identification of misspecification and a robust inference component,
to fit ‘wrong but useful’ models. We propose robust neural posterior estimation
(RNPE), an extension of NPE to simultaneously achieve both these aims, through
explicitly modelling the discrepancies between simulations and the observed
data. We assess the approach on a range of artificially misspecified examples, and
find RNPE performs well across the tasks, whereas naïvely using NPE leads to
misleading and erratic posteriors.