A sequential decomposition of proper scoring rules

The objective assessment of forecasts plays an integral role in the development of a prediction system. Scoring rules condense all information regarding forecast performance into a single numerical value, providing a convenient framework to objectively rank and compare competing prediction schemes. However, the value of a forecast to its user will depend on how it is to be used, and it is therefore necessary to consider several different characteristics of a forecast’s performance. Although scoring rules provide only a single measure of forecast accuracy, they can often be decomposed into components that each assess a distinct aspect of a forecast, such as its calibration or information content. These decompositions of scores provide additional feedback to the forecaster, which can be used to identify strengths and limitations in the prediction scheme, and, in turn, help to improve future forecasts. But these aspects of forecast quality could themselves depend on several factors, such as the time of the year, the spatial location, or on the value of the forecast itself, and it is therefore useful to evaluate the performance of a forecast under different circumstances; if a forecaster were able to identify situations in which their forecasts perform particularly poorly, then they could more easily develop their forecast strategy to account for these deficiencies. To help forecasters identify such situations, we introduce a novel decomposition of scoring rules that allows for a more rigorous examination of the sources of information in a forecast whilst simultaneously quantifying the magnitude of conditional forecast biases. We apply this decomposition to MeteoSwiss COSMO-2E forecasts for the occurrence of moderately extreme weather events and illustrate how the additional information provided by this decomposition can be used to design more appropriate statistical post-processing techniques.