Uncertainty Quantification for Deep Learning

Many processes in the geosciences are highly complex and computationally challenging or  not well known. In those cases, Machine Learning, especially Deep Learning, is becoming increasingly popular to either replace expensive numerical models or parts of those models, or to describe relations between variables where the underlying equations are unknown. Despite many successful applications, the uptake of Deep Learning in science has been slow because proper uncertainty estimates are lacking, while these are crucial for comparison studies, forecasting, and risk management.

Deep Learning can be considered as a method that provides a nonlinear map between an input vector “x” and an output vector “z”. The nonlinear map contains a large weight vector “w,” determined via optimization using training, validation, and testing dataset. To quantify the uncertainty in the output “z”, we need to take into account uncertainty in 1) the input “x”, 2) the weight vector “w”, and 3) the nonlinear map from input to output. Furthermore, the uncertainty in the weight vector depends on uncertainties in the training and testing dataset. Present-day Deep Learning methods such as Bagging, MC Drop-out, and Deep Ensembles ignore most of these uncertainty sources, or apply them incorrectly, resulting in an incorrect uncertainty estimate.

In this presentation, we will, for the first time, take all uncertainty sources into account and provide an efficient methodology to generate output uncertainty estimates. Interestingly, by taking uncertainty in input training data into account, we show that the uncertainty quantification becomes more robust to outliers as it is a systematic and well-defined way to implicitly increase the training dataset. We then demonstrate an application for predicting cloud process rates from a deep neural net and provide a physical interpretation of the resulting uncertainty estimates.