Comprehensive Assessment of Seasonal Forecasts Across Multiple Models and Model Versions

Forecast verification is a well-established activity essential for climate prediction providers to assess and communicate the capabilities and limitations of their forecasts. It is also crucial for informing users about the reliability and usefulness of the climate information they access. However, the verification process involves several methodological choices—such as the selection of reference datasets, spatial transformations, and statistical metrics—which can significantly affect the outcome.

In this study, we use scorecards to summarise and evaluate the impact of key decisions made during forecast verification. Scorecards is a visual synthesis tool we have developed to summarise the performance of seasonal forecast systems across multiple initialisation dates, forecast times, and statistical metrics. Presented in a tabular format, scorecards enable a compact and flexible overview of verification results for user-defined variables, regions, and climate indices, helping to identify consistent patterns without replacing detailed spatial maps. Specifically, we quantify the effects of: (i) regridding forecasts to the observational reference grid versus regridding observations to the model grid, (ii) applying orographic corrections to air temperature when model and observational grids differ, (iii) the timing and implementation of cross-validated anomaly computation, and (iv) the evaluation of forecasts against blended observational products combining air and sea surface temperatures.

We also examine the influence of different approaches to spatially aggregating verification metrics and propose methodologies for assessing the statistical significance of these aggregated scores. By systematically quantifying these methodological impacts, we provide practical recommendations for improving forecast verification practices.

This framework is being tested within the context of the CERISE project, which aims to advance the next generation of C3S seasonal prediction systems through improved land-atmosphere data assimilation and land surface initialisation techniques. Since land surface initial conditions can substantially influence near-surface climate predictions for several months, especially for variables relevant to heatwaves, droughts, and water availability, robust verification is critical. The proposed framework will support fair and consistent comparisons of evolving seasonal forecast systems developed within CERISE.