Bayesian neural networks for metamodeling numerical simulators: toward reliable predictive uncertainty

Overview & Synopsis

Numerical simulation is widely used to model and predict the behavior of complex physical systems. However, some simulators are highly computationally expensive, which limits the number of possible runs and thus their direct use for tasks such as uncertainty propagation or model calibration. A common approach is to replace the simulator with a machine-learning-based metamodel. Gaussian process regression (GPR) is often considered the reference method because it provides both accurate predictions and full predictive distributions. Nevertheless, standard covariance kernels may struggle to capture irregular, discontinuous, or highly non-stationary behaviors. To overcome these limitations, Bayesian neural networks (BNNs) offer a promising alternative as they combine the flexibility and expressive power of neural networks with a Bayesian formulation that naturally yields predictive distributions. Their practical use, however, raises questions regarding prior specification, posterior inference during training, and calibration of model error variance. We propose a heuristic multiobjective procedure for variance calibration and investigate the impact of different prior choices, including Gaussian, hierarchical, and optimized versions informed by GP trajectories. Through analytic test cases, we show that BNNs trained with this procedure can match the predictive accuracy of GPR while providing more reliable uncertainty quantification in some non-stationary settings. Gaussian priors optimized using GPR trajectories, as well as hierarchical priors, appear most effective in these first examples. Finally, we apply BNNs to a severe accident simulator modeling corium-water interaction in a pressurized water reactor, illustrating that these performances hold in higher-dimensional settings and paving the way for more complex applications.