Does AI Learn Physics? Assessing the Physical Fidelity of Data-Driven Tropical Cyclone Forecasts

Machine Learning Weather Prediction (MLWP) models—specifically GraphCast, PanguWeather, Aurora, and FourCastNet—show great promise for competing with physics-based Numerical Weather Prediction (NWP) models by providing global forecasts at a low computational cost. However, a thorough physical evaluation is needed before they can be used in place of NWP models. Our comprehensive study comparing these four leading MLWP models with NWP and observations in Tropical Cyclone (TC) forecasting across all tropical basins uncovers a significant duality: MLWP models are very good at predicting the TC track (with an average error of less than 200 km at a 96-hour lead time) because they accurately capture the underlying dynamics. However, they always underestimate the maximum sustained wind speeds (intensity). This systematic low intensity bias is directly related to biases that come from their ERA5 training data and are made worse by penalties. Even with this limitation, the models accurately depict important physical structures, such as low-level convergence and the vertical warm core, while also keeping different physical fields consistent. This suggests that the models learn how different dynamical and thermodynamical processes are related to each other in a way that makes sense. Ultimately, although MLWPs, especially Aurora, exhibit an implicit comprehension of TC dynamics, their enduring intensity bias requires additional refinement prior to their complete substitution of NWP models.