Evaluating a Pan-European Flood Impact Forecasting System: A Multi-Criteria Framework Integrating Hydrological Skill and End-User Perspectives

Flood forecasting is evolving from predicting hydrological variables to estimating potential impacts, bridging the gap between hazard anticipation and decision-making. Flood impact forecasts are obtained by combining hazard forecasts with the relatively high-resolution exposure datasets; e.g., population density, health and education facilities, transport networks, and energy infrastructure, to support decision-making before and during the event. However, the evaluation of the impact forecast remains challenging. Beyond hydrometeorological forecast skill, a meaningful evaluation of impact forecasts must incorporate ground truth evidence of real-world impacts and feedback from operational end-users.

A Pan-European real-time flood impact forecasting system has been designed within the European project INLINE. The system uses precipitation forecasts generated by seamless blending of probabilistic radar-based nowcasts and the precipitation simulations of the ECMWF EPS (maximum lead time: 120 hours). These are the inputs to estimate the flash flood hazard probabilities throughout Europe, which are integrated with high-resolution open-source exposure datasets to estimate flash flood impacts. INLINE is conducting a 15-month large-scale demonstration with an extensive Community of Interest (COI) including hydrological institutions, civil protection agencies, and emergency managers.

This study presents a multi-criteria evaluation framework applied to assess the performance of the system during the demonstration period. The evaluation integrates four components: (i) Hydrometerological skill, comparing the blended forecast product against radar and gauge observations to evaluate accuracy, reliability and timeliness; (ii) Impact-based verification, evaluating the forecasted impact levels against a newly created real-world impact database, which collects impact information using an LLM-based algorithm through news and social media; (iii) User-centric operational value, quantifying the system’s usefulness, clarity and operational relevance through structured surveys within the COI; and (iv) added value, comparing the complementary of the project developments with the current operational tools used by stakeholders to quantify the improvement for emergency management.

Several representative flood events are analysed in detail to showcase the applicability of the evaluation framework applied to the different developments of the project, and particularly impact-based forecasts. The results underline the importance of combining technical performance metrics with real-world impacts and stakeholder perspectives to guide future operational implementation.