The fractions skill score for ensemble forecast verification

This work explores how the Fractions Skill Score (FSS), originally developed for deterministic forecasts of binary events, can be used for probabilistic forecast verification. By comparing a selection of four ensemble-based methods to compute the FSS, we highlight their distinct behaviour with ensemble size, neighbourhood size, and frequency of occurrence of the forecast event. Our study emphasizes that only a specific variant of the FSS, which we refer to as "probabilistic FSS", demonstrates reasonable behaviour with ensemble size. We reveal that the probabilistic FSS depends on ensemble size in a similar way as the Brier Skill Score, despite performing a neighbourhood-based instead of a grid-point-based forecast evaluation. We derive a formula that describes the expected behaviour of the probabilistic FSS with changes in ensemble size. Finally, utilizing a unique dataset of high-resolution 1000-member ensemble precipitation forecasts for Germany, we explore the impact of ensemble and neighbourhood size on the predictive skill by studying various subsamples of the full ensemble.