Fine-tuning Foundation Models for Benchmarking Prediction Skills for Sub-seasonal Forecasting

Few hundred million to billion parameters of autoregressive transformer-based weather foundation models (FMs) have demonstrated generalizabilities for various downstream applications such as regional forecasting to downscaling. They occasionally outperform traditional physics-based models for medium-range forecasting skills as well as enable significantly faster execution speeds. These ML approaches are designed for timeframes ranging from hourly to up to 10 days, and sub-seasonal forecasting, defined as a range spanning two weeks to two months, often receives less attention for their downstream tasks due to the inherent challenges in predicting the chaotic nature of atmospheric systems. However, the sub-seasonal to seasonal forecast has socio-economic impacts influencing actions from seasonal extreme weather events and economic activities. While community standards for benchmarking studies have been conducted for the medium-range forecasts, the benchmarking of sub-seasonal forecasts still needs further efforts. In this study, we are aiming to fine-tune foundation models to predict sub-seasonal forecasts for various variables to conduct comprehensive benchmarking for weather foundation models. Particularly, to reduce the complexity of tasks, our fine-tune task forecasts two-week averaged atmospheric variables with a forecasting lead-time of two weeks. For this task, we resample the community standard dataset, WeatherBench, for the two-week averaged dataset. We primarily work with the Oak Ridge Base Foundation Model for Earth System Predictability (ORBIT), and extend the benchmarking to other FMs across Aurora, ClimaX, and Prithvi WxC models. Our initial fine-tuning task uses a 100 million parameters ORBIT model to predict geopotential height at 200 hPa with two-week lead time, a key indicator for extreme precipitation in Central Southeast Asia. The preliminary results demonstrate that the fine-tuned ORBIT predicts realistic spatial distributions achieving an MSE of 24.32 m when evaluated against the 2018 data. The comprehensive sub-seasonal forecasting benchmarking can highlight the potential of weather FMs whether they capture underlying principles of atmospheric dynamics, thereby enabling their performance to be extended to longer forecast lead-times.