Evaluation of precipitation forecasts of several NWP modelling systems within South-East European Multi-Hazard Early Warning Advisory System - SEE-MHEWS-A project

SEE-MHEWS-A project aims to provide operational forecasters from 17 National Meteorological and Hydrological Services of the South-East Europe with effective and tested tools for forecasting hazardous weather events and their possible impacts in order to improve the accuracy of warnings and their relevance to stakeholders and users. On a single virtual platform, the system collects NWP information for the provision of accurate forecasts and warnings to support hazard-related decision-making by national authorities. In addition to the operational IFS model of the European Centre for Medium-Range Weather Forecasts (ECMWF), four limited area model (LAM) configurations for South-East Europe (SEE) are developed and run in a quasi-operational environment including: ALADIN-ALARO, COSMO, ICON, and WRF NMM-B. Our interest is a structured, region-wide forecast verification as a necessary component of the process to establish a multi-hazard early warning system.

We study strengths and weaknesses of different modelling systems in different weather situations to provide an initial estimate of model success in a real-time operational situation for precipitation, including extreme events, over a pilot area of wider eastern Adriatic coast spanning over five countries. First, our approach uses moment-based and categorical statistical verification, including a decomposition of scores into biases and dispersion (phase) errors. Best results averaged over all stations are not achieved for a single model but vary across several modelling configurations depending on the score analyzed. It is clearly noticeable that models are of lower accuracy near the mountainous coast compared to continental inland due to the generally more intense precipitation, influence of the complex terrain and influence of sea and surface inhomogeneities. Likewise, categorical verification suggests low-medium intensity precipitation forecast accuracy is the lowest where Dinaric Alps are most complex, but results do improve in higher precipitation categories. Although results are far from perfect for most extreme events, all models show skillful predictions and none of the models shows considerably more strengths than others with extremal dependence index (EDI) ranging from 0.45 up to 0.85 depending on the model.

To alleviate effects of small errors in time and space on verification measures, in absence of spatially homogenous precipitation data, we apply a neighborhood verification approach, which offers an alternative that rewards closeness by relaxing the requirement for exact matches between forecasts and observations in the spatial domain. The single-observation neighborhood approach (SO-NF) results show an improvement in ETS values with an increase of the spatial scale for the categories of events. At the highest precipitation category (above 30 mm/24h) and common spatial scale of ~45 km, ECMWF and COSMO models seem to perform somewhat more consistently than other models. Nevertheless, the improvement of the results with the forecast neighborhood size noticed for most models and countries shows the benefits of the SO-NF approach in terms of recognizing additional forecasted values present in the proximity of the exact location, even though they were slightly displaced.