Precipitation verification as an important task of South-East Multi-Hazard Early Warning Advisory System project

The main goal of the South-East Multi-Hazard Early Warning Advisory System project (SEE-MHEWS-A) is to provide support for the National Meteorological and Hydrological Services in Southeast Europe to produce timely and accurate warnings of hazardous weather and hydrological events. The reliability of such a system largely depends on the adequate performance of numerical weather predictions (NWP). Therefore, an adequate verification procedure applied to several NWPs region-wide is a necessary component of the process for building such a system.

The verification methodology consists of several building blocks, starting with the analysis of the missing observations and climatology analysis. After the preparatory steps, the methodology includes the verification of a continuous predictand, engaging several conventional verification measures such as Pearson correlation coefficient, systematic error, mean absolute error and root mean square error. Additionally, the verification of a categorical predictand is performed, using several thresholds to describe different precipitation intensities. For climatologically common events the measures such as frequency bias, hit rate,  false alarm ratio and equitable threat score are used to evaluate the forecast quality. The extremal dependence index is used to access the performance for rare events. Finally, the single observation - neighborhood forecast (SO-NF) verification approach is engaged. This approach allows for a more fair comparison between models of different resolutions by comparing results for similar spatial scales.

The five numerical modeling systems available for verification include ALADIN-ALARO (Aire Limitée Adaptation dynamique Développement InterNational), COSMO (Consortium for Small-scale Modeling), ECMWF-IFS (Integrated Forecast System), ICON (Icosahedral Nonhydrostatic) and NMM-B (Nonhydrostatic Multi-scale Model) models. The verification analysis is done on a domain that includes several countries in southeast Europe for a 24-h cumulative precipitation variable. The area tested includes a diversity in orography, which contributes to a better assessment of the performance of the NWP models being evaluated. 

Results often indicate moderately or strongly correlated data and bias mostly pronounced in the areas of the complex orography. The error increases from more flatter areas towards complex ones. The RMSE decomposition shows that dispersion error is the predominant source of error, while systematic sources of error are considerably smaller for all forecasts tested. Regarding categorical verification, all models produce excellent results for the dry day category, while there is usually lower quality for the remaining precipitation events. The SO-NF approach indicates that there are useful additional forecasts present in the proximity of the exact location, even though they are slightly displaced, showing the benefits in terms of better assessment of forecast quality for a rare event.

Finally, since all models do show specific benefits and limitations, verification results suggest that a multi-model ensemble might be a step further for the exploiting full predictive potential of these systems.