A user-relevant approach to the verification of flood forecasts

Implementing flood forecasting models and early warning systems in decision-making can substantially reduce the impacts of floods and other natural hazards and provide more time for warnings and anticipatory action. Verification of flood forecasts is critical in providing emergency managers with practical information about the forecasts’ performance. However, incorporating forecasts in decision-making and warning applications demands a user-relevant categorical verification approach that covers practical aspects of operational systems. In this regard, standard scientific practice for verification may not match the forecast users’ experience or be suitable for all end users. It creates several challenges in categorical verification, including the vague definitions of the 2×2 contingency table categories, as they do not consider the slightly misaligned timing of events. Another challenge is the counting method of the categories, where different counting methods, based on a regular time-step basis, or based on individual events, may lead to different conclusions, and therefore, deliver different performance insights and value for forecast users. This research addresses these challenges, proposing a user-relevant verification approach, and examining the corresponding effects on the performance quality and economic value of flood forecasts. Two global forecasting models –Google AI model and GloFAS– are verified with the proposed approach in the African continent. Our results show a consistent pattern of decrease in performance when considering a user-relevant approach in comparison with standard verification practice. These findings emphasize a gap between standard and user-tailored verification, and the need for user-relevant verification of other flood forecasting systems, with the consideration of implementing additional aspects of operational systems.