Verification of the weather forecast of seven weather forecast providers for 500 locations worldwide

The numerical weather forecast has been continuously improved within the last decades due to a) improved subgrid parameterisations, b) more precise initial conditions (e.g., satellite imagery) and c) more computational power allowing a finer grid resolution.  

Weather forecast providers typically rely on raw weather forecast models on the market in combination with own developed post-processing routines including e.g. weather stations and AI techniques to increase the quality of the forecast.  

In this study the weather forecast of seven different weather forecast providers was verified at 500 measurement locations worldwide for the year 2022. The analysis was conducted for air temperature, precipitation, wind speed and direction and air pressure for the forecast period 1 – 6 days. Forecast data were verified against METAR measurements in hourly temporal resolution.  

Error metrics such as mean absolute error (MAE), mean bias error (MBE), Pearson correlation, Heidke skill score (HSS), Probability of detection (POD) and False alarm rate (FAR) have been calculated for all weather forecast providers. 

The MAE for the 12 – 35 h air temperature forecast of the seven different weather forecast providers ranges between 1.26 K and 2.06 K, whereas three providers showed values below 1.5 K. The percentage of MAE values lower than 1.5 K in the 1-day forecast ranges between 67 % and 35 %, showing a high difference in the post-processing routines of the different weather forecast provider.  

The forecast quality is lowered with increasing forecast horizons due to the physical limitations of the weather forecast. For example, the best provider for the 12 – 35 h air temperature forecast showed a MAE of 1.26 K for the first day, 1.34 for the second day and 1.43 for the third day. This results additionally shows that the 3-day air temperature forecast of the best provider is even better than the 1-day air temperature forecast of 5 (out of 7) analysed weather forecast providers.  

The results in this study show a broad quality of weather forecast providers, indicating that some providers rely on simple raw weather forecast models, whereas other provider lower the errors by post processing routines based on AI techniques and additional weather stations.