Enhanced Foundation Model through Efficient Finetuning for Extended-Range Weather Prediction

Weather forecasting is a vital topic in meteorological analysis, agriculture planning, disaster management, etc. The accuracy of forecasts varies with the prediction horizon, spanning from nowcasting to long-range forecasts. The extended range forecast, which predicts weather conditions beyond two weeks to months ahead, is particularly challenging. This difficulty arises from the inherent variability in weather systems, where minor disturbances in the initial condition can lead to significantly divergent future trajectories.

Numerical Weather Prediction (NWP) has been the predominant approach in this field. Recently, deep learning (DL) techniques have emerged as a promising alternative, achieving performance comparable to NWP [1, 2]. However, their lack of embedded physical knowledge often limits their acceptance within the research community. To enhance the trustworthiness of DL-based weather forecasts, we explore a transformer-based framework which considers complex geospatial-temporal (4D) processes and interactions. Specifically, we select the Pangu model [3] with a 24-hour lead time as the initial framework. To extend the prediction horizon to two weeks ahead, we employ a low-rank adaptation for model finetuning, which saves computation resources by reducing the number of parameters to only 1.1% of the original model. Besides, we incorporate multiple oceanic and atmospheric indices to capture a broad spectrum of global teleconnections, aiding in the selection of important features.

Our contributions are threefold: first, we provide an operational framework for foundation models, improving their applicability in versatile tasks by enabling training rather than limiting them to inference stages. Second, we demonstrate how to leverage these models with limited resources effectively and contribute to the development of green AI. Last, our method improves performance in extended-range weather forecasting, offering enhanced prediction skills, physical consistency, and finer spatial granularity. Our methodology achieved reduced RMSE on T2M, Z500, and T850 for 0.13, 139.2, and 0.52, respectively, compared to IFS. In the future, we plan to explore other settings, such as predicting precipitation and extreme temperatures.

REFERENCES
[1] Nguyen, Tung, et al. "ClimaX: A foundation model for weather and climate." arXiv preprint arXiv:2301.10343 (2023).
[2] Lam, Remi, et al. "Learning skillful medium-range global weather forecasting." Science (2023): eadi2336.
[3] Bi, Kaifeng, et al. "Accurate medium-range global weather forecasting with 3D neural networks." Nature 619.7970 (2023): 533-538.