UNITE: A toolbox for unified diagnostic evaluation of physics-based, data-driven and hybrid models based on information theory

The use of “hybrid” models that combine elements from physics-based and data-driven modelling approaches has grown in popularity and acceptance in recent years, but these models also present a number of challenges that must be addressed in order to ensure their effectiveness and reliability. In this project, we propose a toolbox of methods based on information theory as a step towards a unified framework for the diagnostic evaluation of “hybrid" models. Information theory provides a set of mathematical tools that can be used to study input data, model architecture and predictions, which can be helpful in understanding the performance and limitations of “hybrid” models.

Through a comprehensive case study of rainfall-runoff hydrological modelling, we show how a very simple physics-based model can be coupled in different ways with neural networks to develop “hybrid” models. The proposed toolbox is then applied to these “hybrid” models to extract insights which guide towards model improvement and refinement. Diagnostic scores based on the entropy (H) of individual predictions and the Kullback-Leibler divergence (KLD) between predictions and observations are introduced. Mutual information (I) is also used as a more all-encompassing metric which informs on the aleatory and epistemic uncertainties of a particular model. In order to address the challenge of calculating quantities from information theory on continuous variables (such as streamflow), the toolbox takes advantage of different estimators of differential entropy, namely: binning, kernel density estimation (KDE) and k-nearest neighbors (k-NN).