Modulating Surrogates for Bayesian Optimization

Erik Bodin, Markus Kaiser, Ieva Kazlauskaite, Zhenwen Dai, university bath, Carl Henrik Ek

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

1 Downloads (Pure)

Abstract

Bayesian optimization (BO) methods often rely on the assumption that the objective function is well-behaved, but in practice, this is seldom true for real-world objectives even if noise-free observations can be collected. Common approaches, which try to model the objective as precisely as possible, often fail to make progress by spending too many evaluations modeling irrelevant details. We address this issue by proposing surrogate models that focus on the well-behaved structure in the objective function, which is informative for search, while ignoring detrimental structure that is challenging to model from few observations. First, we demonstrate that surrogate models with appropriate noise distributions can absorb challenging structures in the objective function by treating them as irreducible uncertainty. Secondly, we show that a latent Gaussian process is an excellent surrogate for this purpose, comparing with Gaussian processes with standard noise distributions. We perform numerous experiments on a range of BO benchmarks and find that our approach improves reliability and performance when faced with challenging objective functions.
Original languageEnglish
Title of host publication37th International Conference on Machine Learning, ICML 2020
PublisherML Research Press
Pages947-956
Number of pages10
Volume119
ISBN (Electronic)9781713821120
Publication statusPublished - 13 Jul 2020
Event37th International Conference on Machine Learning (ICML 2020) -
Duration: 12 Jul 202018 Jul 2020

Conference

Conference37th International Conference on Machine Learning (ICML 2020)
Period12/07/2018/07/20

Bibliographical note

International Conference on Machine Learning (ICML 2020)

Keywords

  • stat.ML
  • cs.LG

Fingerprint

Dive into the research topics of 'Modulating Surrogates for Bayesian Optimization'. Together they form a unique fingerprint.

Cite this