EMS Annual Meeting Abstracts
Vol. 18, EMS2021-336, 2021
https://doi.org/10.5194/ems2021-336
EMS Annual Meeting 2021
© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

A sequential decomposition of proper scoring rules

Sam Allen1, Chris Ferro2, and Frank Kwasniok2
Sam Allen et al.
  • 1Institute of Mathematical Statistics and Actuarial Science, University of Bern, Switzerland (sam.allen@stat.unibe.ch)
  • 2University of Exeter, College of Engineering, Maths and Physical Sciences, Department of Mathematics, United Kingdom of Great Britain – England, Scotland, Wales

The objective assessment of forecasts plays an integral role in the development of a prediction system. Scoring rules condense all information regarding forecast performance into a single numerical value, providing a convenient framework to objectively rank and compare competing prediction schemes. However, the value of a forecast to its user will depend on how it is to be used, and it is therefore necessary to consider several different characteristics of a forecast’s performance. Although scoring rules provide only a single measure of forecast accuracy, they can often be decomposed into components that each assess a distinct aspect of a forecast, such as its calibration or information content. These decompositions of scores provide additional feedback to the forecaster, which can be used to identify strengths and limitations in the prediction scheme, and, in turn, help to improve future forecasts. But these aspects of forecast quality could themselves depend on several factors, such as the time of the year, the spatial location, or on the value of the forecast itself, and it is therefore useful to evaluate the performance of a forecast under different circumstances; if a forecaster were able to identify situations in which their forecasts perform particularly poorly, then they could more easily develop their forecast strategy to account for these deficiencies. To help forecasters identify such situations, we introduce a novel decomposition of scoring rules that allows for a more rigorous examination of the sources of information in a forecast whilst simultaneously quantifying the magnitude of conditional forecast biases. We apply this decomposition to MeteoSwiss COSMO-2E forecasts for the occurrence of moderately extreme weather events and illustrate how the additional information provided by this decomposition can be used to design more appropriate statistical post-processing techniques.

How to cite: Allen, S., Ferro, C., and Kwasniok, F.: A sequential decomposition of proper scoring rules, EMS Annual Meeting 2021, online, 6–10 Sep 2021, EMS2021-336, https://doi.org/10.5194/ems2021-336, 2021.

Displays

Display file

Supporters & sponsors