Fair logarithmic score for multivariate Gaussian forecasts

In evaluating multivariate probabilistic forecasts predicting vector quantities such as a weather variable at multiple locations or a wind vector, an important step is the assessment of their calibration and reliability. Here, we focus on the logarithmic score and are interested in the specific case when the density is multivariate normal with mean and covariance structure given by the ensemble mean and ensemble covariance matrix, respectively. Under the assumptions of multivariate normality and exchangeability of the ensemble members, a relationship is derived that describes the dependence on ensemble size. It is exploited to introduce a fair logarithmic score for multivariate ensemble forecasts [1].

An application to medium-range weather forecasts demonstrates the usefulness of the ensemble size adjustments when multivariate normality is only an approximation, where we consider ensemble predictions of sizes from 8 to 100 of vectors consisting of several different combinations of upper air variables. We show how the logarithmic score depends on ensemble size for various examples and to what extent the fair logarithmic score reduces this dependence.

References

1. Leutbecher, M. and Baran, S., Ensemble size dependence of the logarithmic score for forecasts issued as multivariate normal distributions. Q. J. R. Meteorol. Soc. 151 (2025), paper e4898, doi:10.1002/qj.4898.

*Research was supported by the Hungarian National Research, Development and Innovation Office under Grant No. K142849.