Fair Box ordinate transform for multivariate Gaussian forecasts

In evaluating multivariate probabilistic forecasts predicting vector quantities such as a weather variable at multiple locations or a wind vector, an important step is the assessment of their calibration and reliability. Here, we focus on the Gaussian Box ordinate transform (BOT), which is appropriate if the forecasts and observations are multivariate normal. The BOT is based on the Mahalanobis distance of the observation vector and the estimated Gaussian mean and asymptotically standard uniform if the forecasts and the observation are drawn from the same multivariate Gaussian law. However, for small ensemble sizes combined with high dimensionality, deviation from uniformity is substantial even for reliable forecasts, resulting in hump-shaped or triangular BOT histograms. To circumvent this problem, we derive an ensemble size and dimension-dependent fair version of the Gaussian BOT, where the uniformity holds for any combination of these parameters. With the help of a simulation study, first, we assess the behaviour of the fair BOT for various dimensions, ensemble sizes, and types of calibration misspecification. Then, using ensemble forecasts of vectors consisting of multiple combinations of upper-air weather variables, we demonstrate the usefulness of the fair BOT when multivariate normality is only an approximation.

*Research was supported by the Hungarian National Research, Development and Innovation Office under Grant No. K142849.