Bayesian updating despite model errors? A sliding time-window approach to rescue

Bayesian mechanistic modeling often suffers from overconfident and biased posterior distributions for parameters and predictions. This phenomenon arises because the fundamental assumption of Bayesian Model Analysis is violated: the underlying model is assumed to be true, but in fact, it is a simplification of reality with structural errors that show at least during some periods of the modeled time span. As a result, a compromise solution in parameter space is identified that can formally fit the full data set best, but this parameter set will not be representative of the true system state. Neither will it be representative of the “compensation mode” in which the model is whenever structural error kicks in. As a logical consequence, predictions will be biased and their intervals too narrow. The longer the data set used for calibration, the stronger the misleading effect. Typical sources of severe structural deficits that produce dynamically occurring errors are missing or misspecified processes in the model. 

We propose a formal time-windowed Bayesian analysis to overcome this general problem. When performing Bayesian updating on shorter time windows, the assumption of a (quasi-) true model becomes more plausible, and by sliding this window through the calibration time series, we let the model adjust its posterior parameter distributions according to the current strength of error. These time-shifting parameter distributions allow us to (1) identify periods of statistically significant model error occurrence via measuring time-varying Bayesian model evidence, (2) diagnose potential sources of model error by understanding the time-varying parameter compensation mechanisms, and (3) predict with more realistic uncertainty intervals by distribution averaging. 

We demonstrate the proposed method on a set of synthetic and real-world scenarios of soil moisture modeling. With this example, we also highlight its usefulness to analyze dynamic systems in a wide range of disciplines, such as water quality modeling, decision support, and risk assessment. Results show that the time sequence of posterior parameter distributions (and dependent model mechanisms such as water retention curves and unsaturated hydraulic conductivity functions) provides valuable insights into the model’s weaknesses, and it also provides guidance for model improvement.