On the reliability of bivariate forecasts

Reliability is a key attribute of an ensemble forecast. Typically, this means that one expects that the ensemble spread reflects the potential error of the corresponding ensemble mean forecast. In the realistic case of an unperfect forecast, reliability deficiencies can be diagnosed with tools such as the reliability diagram and the rank histogram. Along with the computation of scores, the use of these diagnostic tools is common practice in ensemble forecast verification when assessing univariate forecasts. But what does reliability mean in practical terms when assessing multivariate forecasts? Here the concept of reliability is revisited in the simplest of the multivariate cases: the bivariate forecast. As a result, we propose a set of new diagnostic tools with an emphasis on the cross-variable reliability aspect. Real case examples are used for illustration and discussion.