Global verification of the 24-hour precipitation forecast of numerical weather prediction models

Verification of precipitation forecasts are rarely published in scientific literature. This study deals with a verification of the 24-hour-precipitation forecast of the global numerical weather prediction models GFS, ICON, ARPEGE and NEMS, as well as the reanalysis model ERA5.

For model comparison more than 10’000 precipitation measurements worldwide from the measurement network METAR were used in hourly resolution. Annual, monthly, and daily precipitation sums of the global prediction models were compared with quality-controlled measurements for the year 2020. Continuous error metrics (e.g., mean absolute errors (MAE), mean bias errors (MBE)) as well as categorical error metrics (e.g., probability of detection, false alarm rate, Heidke Skill Score) are calculated for each measurement location separately, as well as averaged by a 2 x 2 degrees grid to account for the unequal global distribution of the measurements.

In general, the model errors are large in equatorial regions, areas close to the coast and in mountainous areas, where the annual precipitation amount is typically large. The best model performance was analysed for the reanalysis model ERA5 with a mean absolute error of 234 mm and a small mean bias error of -6 mm. ICON showed the lowest MAE of the operational weather forecast models (MAE = 253 mm) followed by GFS (MAE = 270 mm), ARPEGE (MAE = 281 mm) and NEMS (MAE = 296 mm). NEMS and ICON tend to underestimate annual precipitation amounts, whereas GFS and ARPEGE tend to overestimate annual precipitation amounts. Daily precipitation events larger than 1 mm were detected most accurate with ICON (HSS = 0.46) and ERA5 (HSS = 0.45), followed by GFS, ARPEGE and NEMS.

For each country, the model with the highest accuracy was determined based on the MAE of annual precipitation sums and based on the accuracy of daily precipitation events. For more than 80 % of all countries worldwide ERA5 or ICON showed the best model performance.