Hierarchical Testing of a Hybrid Machine Learning-Physics Global Atmosphere Model

Machine learning (ML)-based models have recently demonstrated high skill and computational efficiency, often outperforming conventional physics-based models in weather forecasting and subseasonal prediction. While prior efforts have assessed their ability to capture atmospheric dynamics at the synoptic scale, their performance across broader timescales and under out-of-distribution forcing remains insufficiently understood but essential criterion for establishing their credibility in Earth system science.

In this study, we design three idealized test cases to evaluate the Neural General Circulation Model (NeuralGCM), a hybrid model that couples a dynamical core with ML-based physical parameterizations. The test casts span synoptic-scale phenomena, interannual variability, and out-of-distribution forcings via uniform warmings. We benchmark NeuralGCM against observations and conventional physics-based Earth system models (ESMs). At the synoptic scale, NeuralGCM captures the evolution and propagation of extratropical cyclones with performance comparable to ESMs. At the interannual scale, when forced by El Niño-Southern Oscillation sea surface temperature (SST) anomalies, NeuralGCM successfully reproduces associated teleconnection patterns but exhibits deficiencies in capturing nonlinear response. Under out-of-distribution uniform-warming forcings, NeuralGCM simulates similar responses in global-average temperature and precipitation and reproduces large-scale tropospheric circulation features similar to those in ESMs. Notable weaknesses include overestimating the tracks and spatial extent of extratropical cyclones, and biases in the teleconnected wave train triggered by tropical SST anomalies. Furthermore, its simulated temperature responses near the tropopause and in the stratosphere under uniform warming simulations deviate from those in physics-based models, likely due to the biases in vertical temperature advection by the residual circulation.

Despite these limitations, NeuralGCM exhibits credible responses across all test cases and performs comparably to both observations and physics-based ESMs. These results suggest that hybrid models like NeuralGCM, which integrate dynamical cores with ML physics, offer a promising path toward the next generation of ML-based ESMs.