Evaluating the Performance of Flood Forecasting Models in Japan (2020-2024): Insights from Today's Earth, MLIT Observations and Google's SOTA hydrological model

Flood forecasting models are essential tools for mitigating the impacts of extreme hydrological events by providing early warnings and actionable insights. This study evaluates the performance of the Today's Earth (TE) model, JAXA's land surface simulation system developed under joint research with the University of Tokyo, by comparing its predictions against observed water level data from the Ministry of Land, Infrastructure, Transport and Tourism (MLIT) measuring stations in Japan. The study also compares the results to Google’s SOTA hydrological model using the Google Runoff Reanalysis & Reforecast (GRRR) dataset. The GRRR model features a 7-day (168-hour) forecast window, while Today's Earth offers a shorter forecast window of 39 hours. The analysis focuses on major flood events in Japan (2020–2024), including typhoons and heavy precipitation events, to examine trends and accuracy in flood predictions over different lead times. This evaluation identifies strengths and areas for improvement in operational forecasting across diverse hydrological scenarios.

The methodology integrates a comprehensive dataset of water level observations and forecast outputs, with an emphasis on lead time-dependent peak timing and magnitude error. Forecasts were evaluated based on their ability to capture observed flood peaks, with errors in both peak magnitude and timing quantified for varying lead times. Performance metrics such as Nash-Sutcliffe Efficiency (NSE), Kling-Gupta Efficiency (KGE), and Pearson correlation were calculated for each location and lead time.

Results reveal that forecast accuracy varies significantly with lead time, playing a critical role in the effectiveness of early warning systems. While Google's hydrological model performs better under normal flow conditions, Today's Earth model performs considerably better in both peak magnitude and timing during flood events. The GRRR dataset consistently underestimated peak magnitudes, highlighting the difficulty of forecasting extremes. Another key finding is the marked improvement in forecasting accuracy for the Today’s Earth model between 2021 and 2022, which coincides with the incorporation of observed precipitation data. Initial results indicate a significant enhancement in peak flow timing predictions following this update. This study evaluates how this modification improved forecasting results, emphasising the potential to refine TE’s algorithms and integrate additional observational data.

This research provides actionable insights into flood prediction reliability and demonstrates the value of leveraging Japan’s extensive network of water level gauges. Findings contribute to ongoing efforts to enhance flood forecasting systems globally and highlight the importance of targeted evaluations for improving model performance. The study's implications extend to disaster risk management, operational forecasting practices, and the broader pursuit of climate-resilient water management strategies.