A generalisation of the signal-to-noise ratio using proper scoring rules

A signal-to-noise "paradox" was first described in the context of ensemble forecasts on seasonal timescales. It refers to a situation in which the correlation between the ensemble mean and the actual verification is larger than the correlation between the ensemble mean and individual ensemble members. A noted problem of the signal-to-noise paradox remains that the signal-to-noise ratio itself, or equivalently the ratio of predictable components (RPC), which are used to diagnose the signal-to-noise paradox, has poorly understood statistical properties, rendering reliable identification of the signal-to-noise paradox difficult.

In this contribution, a generalised concept of the RPC is discussed based on proper scoring rules. This definition is the natural generalisation of the classical RPC, yet it allows one to define and analyse the signal-to-noise properties of any type of forecast that is amenable to scoring, thus drastically widening the applicability of these concepts. The methodology is illustrated for ensemble forecasts, scored using the continuous ranked probability score (CRPS), and for probability forecasts of a binary event, scored using the logarithmic score. Numerical examples demonstrate that the classical and new RPC statistic agree regarding which data sets exhibit anomalous signal-to-noise ratios, but exhibit different variance, indicating different statistical properties.