Spatial aggregation of ROC and PR curves

Receiver Operating Characteristic (ROC) and Precision–Recall (PR) curves are widely used to assess the discrimination ability of forecasts for binary events, such as threshold exceedances or warnings of extreme events. In weather forecasting, forecasts are provided as spatial fields, yielding location-wise ROC and PR curves that are often aggregated to facilitate comparison, although the effect of the aggregation strategy on performance assessment remains poorly understood.

We investigate how different aggregation strategies for ROC and PR curves affect the assessment of discrimination ability. In particular, we identify conditions under which aggregation strategies satisfy two desirable properties for fair comparison: preservation of dominance between forecasts and preservation of concavity of the curves. We review commonly used aggregation approaches from the literature, analyze their theoretical properties, and highlight potential pitfalls that may lead to misleading interpretations. Based on these findings, we provide practical guidelines for the interpretation of aggregated ROC and PR curves. The proposed framework is illustrated using AI-based global weather forecasts, showing how different aggregation strategies can lead to different rankings.