Can AI-based weather prediction models simulate the butterfly effect?

Recently, weather prediction models based on artificial intelligence (AI) have become equally to slightly more accurate than the established operational weather prediction models in terms of deterministic scores. The much lower computational cost of the AI-models facilitates the generation of large ensembles, hence it is important to assess if their error growth properties are realistic. Here, we investigate error growth from initial condition perturbations with varying amplitudes, simulated with the AI-weather prediction models (PANGU, GraphCast, FourCastNet) and with a “classic” fluid equation-based weather model (ICON). From past research and the global convection-permitting ICON simulations, it is expected that small-amplitude initial condition perturbations would grow very fast initially in areas with latent heat release, then spreading out to larger and larger scales, eventually setting a fixed and fundamental limit to the predictability of weather. This phenomenon is known as the butterfly effect. We find however, that in contrast to ICON, the AI-based models completely fail to reproduce the rapid initial growth rates. Instead their growth rates remain similar to those typically found on synoptic-scales, which incorrectly suggests an unlimited predictability of the atmosphere. In contrast, if the initial perturbations are large in amplitude and comparable to current uncertainties in the estimation of the initial state, the AI-based models basically agree with results from the “physically-based” ICON simulations, although some deficits are still present, mostly related to their particularly low effective resolution. This provides an example of how machine learning models can fail to reproduce a fundamental physical principle, even though they can accurately mimic many observed behaviors.