Huge Ensembles of Weather Extremes using the Fourier Forecasting Neural Network

Studying low-likelihood high-impact extreme weather and climate events in a warming world requires massive
ensembles to capture long tails of multi-variate distributions. In combination, it is simply impossible to generate
massive ensembles, of say 10,000 members, using traditional numerical simulations of climate models at high
resolution. We describe how to bring the power of machine learning (ML) to replace traditional numerical
simulations for short week-long hindcasts of massive ensembles, where ML has proven to be successful in terms of
accuracy and fidelity, at five orders-of-magnitude lower computational cost than numerical methods. Because
the ensembles are reproducible to machine precision, ML also provides a data compression mechanism to
avoid storing the data produced from massive ensembles. The machine learning algorithm FourCastNet (FCN) is
based on Fourier Neural Operators and Transformers, proven to be efficient and powerful in modeling a wide
range of chaotic dynamical systems, including turbulent flows and atmospheric dynamics. FCN has already been
proven to be highly scalable on GPU-based HPC systems. 

We discuss our progress using statistics metrics for extremes adopted from operational NWP centers to show
that FCN is sufficiently accurate as an emulator of these phenomena. We also show how to construct huge
ensembles through a combination of perturbed-parameter techniques and a variant of bred vectors to generate a
large suite of initial conditions that maximize growth rates of ensemble spread. We demonstrate that these
ensembles exhibit a ratio of ensemble spread relative to RMSE that is nearly identical to one, a key metric of
successful near-term NWP systems. We conclude by applying FCN to severe heat waves in the recent climate
record.