Operational Application and Performance Evaluation of AI-based Weather Prediction Models: Impacts of Initial Conditions and Heavy Rainfall Biases

The Korea Meteorological Administration (KMA) has been operating GraphCast, Pangu-Weather, and FourCastNet based on initial conditions from the Korean Integrated Model (KIM), the Unified Model (UM), and the ECMWF model since 2024.

A comparative evaluation between AI-based and NWP-based forecasts indicates that, when the same initial conditions are used, GraphCast and Pangu-Weather generally outperform traditional NWP models. While AI forecasts initialized with ECMWF analyses outperformed the ECMWF NWP model, AI forecasts initialized with KIM analyses do not surpass the performance of the ECMWF NWP system, highlighting the critical role of initial condition quality. These results demonstrate that, despite rapid advancements in AI models, forecast skill remains strongly dependent on the accuracy of the driving initial fields.  Therefore, improving the quality of initial conditions through continued advancement of NWP systems is essential for maximizing the performance of AI-based forecasts.  
In addition, the superior performance of AI forecasts initialized with ECMWF analyses may not be solely attributable to the higher quality of the initial conditions. It is also likely influenced by the fact that AI models have been trained on datasets generated by the ECMWF model, implying that consistency between the training data and the initial conditions plays a significant role in enhancing forecast skill.  

Several limitations of AI models have become evident through years of operational use. In particular, during summer heavy rainfall events, AI models tend to underestimate precipitation intensity, as has been widely demonstrated in numerous previous studies. To enhance the operational applicability of AI-based forecasts, it is necessary to develop post-processing and bias-correction techniques that mitigate such underestimation. A synergistic approach combining AI model development, NWP improvement, and targeted bias correction is expected to further advance forecast accuracy and reliability in operational settings.

Keywords: AI Weather Forecasting, GraphCast, Pangu-Weather, Initial Conditions, KIM, Precipitation Underestimation, bias correction