Decision-oriented benchmarking of AI weather models for subseasonal monsoon onset forecasts in India

Rapid advances in artificial intelligence weather prediction (AIWP) have enabled AI models to potentially outperform traditional numerical weather prediction (NWP) models while requiring only a fraction of the computational resources. However, many AI forecast evaluation studies have compared models using global metrics over limited years without focusing on sector and region-specific applications. Operationally driven benchmarking is necessary to effectively deploy these models, informing both model selection and improvements for different decision-making needs. Such benchmarking has been instrumental in driving AI progress in areas like ImageNet and AlphaFold. In this work, we benchmark the performance of six state-of-the-art AIWP models (AIFS, FuXi, FuXi-S2S, GraphCast, GenCast, NeuralGCM) and an NWP model (IFS) in forecasting local-scale agriculturally relevant monsoon onset over India. The models’ onset forecasts are compared with over a century of rain gauge–based ground truth observations, using standard verification metrics for both deterministic and probabilistic forecasts. This multiperiod evaluation is specifically designed to align with how such forecasts will be disseminated to stakeholders. In this operationally oriented benchmarking, we find that most AIWP models outperform the climatological baseline forecasts at medium-range timescales (~15 days), but exhibit comparable skill at subseasonal timescales (~30 days) in the core monsoon zone. These models also achieve comparable performance to IFS, while enabling calibration of probabilistic forecasts through precisely controlled ensembles that can be efficiently generated for multiple past decades. The speed and open-source nature of AIWPs provide the additional advantage that one can localize such models. 

This benchmark guided model selection for large-scale AI-based generation and dissemination of the 2025 monsoon onset forecast to 38 million farmers in India. Our work presents a framework for developing operational, decision-oriented benchmarks that can accelerate the translation of the AI-driven second weather revolution into the democratization of weather forecasting worldwide.