Quantification of 6-hour precipitation field smoothing in deterministic Machine Learning-based Weather Forecasts

Starting in the early 2020s, artificial intelligence-based models (Machine Learning Weather
Prediction, MLWP) play an increasingly important role in weather forecasting. Their widely known
advantages make the use of MLWP models desirable in the future; however, it is important to have
a thorough understanding of the nature of their prediction fields and their strengths and weaknesses.
Most deterministic MLWP models have been trained using a Mean Squared Error (MSE)-
based cost function. The effective resolution of these models decreases as the forecast time
increases to avoid the double penalty occurring in the evaluation of sharp precipitation fields. It has
been shown that the effective resolution of a MLWP model forecast field is equivalent to the
ensemble average for forecast time period which had been explicitly included in the model cost
function. Since this similarity to the ensemble mean provides a strong basis for the operational
interpretation of the MLWP forecasts, it is important to verify it using model data for different
atmospheric variables.
In our work, we examined the effective resolution of a particularly important variable in
forecasting practice, the 6-hour precipitation field, by comparing the ECMWF-AIFS MLWP model,
the ECMWF HRES ensemble average, and the ERA5 reanalysis data. Effective resolution is usually
determined by examining the spectrum of model fields. However, since precipitation is a highly
localized and non-continuous atmospheric variable, we considered it appropriate to quantitatively
examine the smoothing of the fields in addition to spectral analyses. The aim of our calculations
was to determine how long the smoothing of ECMWF-AIFS precipitation forecasts follows the
ensemble average and how much they deviate from reality as represented by ERA5. We defined
smoothing as the extent to which the ERA5 precipitation field, which most accurately represents the
real field, needs to be smoothed using Gaussian filters in order for its sharpness indicators to match
those of the given forecast. Since smoothing in the precipitation field of MLWP models occurs not
only in the spatial structure but also in the reduction of extremes, we have developed a separate
method for examining sharpness that focuses exclusively on extreme values, based on the
examination of the exponential approximation of the global distribution of 6-hour precipitation
intensity.
Our results show that the agreement between the MLWP model and the ensemble mean
smoothing over the training period is only achieved in the case of extratropical precipitation
dominated by synoptic-scale processes. We did not observe such an agreement at all in tropical
areas with predominantly convective precipitation. We obtained similar results using a sharpness
metric developed for extreme values, i.e., the MLWP model's smoothing of extreme values only
matched the ensemble mean in the extratropics These results indicate that MLWP models can only
successfully predict processes on a sufficiently large scale and smooth small-scale processes such as
convection.