Evaluation of three short-range (0-6h) rain ensemble forecasts: study of the Aude October 2018 flash floods (southeastern France)

The Aude river flash floods that occurred on October 15 and 16, 2018 are among the most important in southeastern France in recent years. The triggering rainfall of the event was characterized by a very fast evolution and low predictability. During the night, nearly 243.5mm of rain in 6 hours was recorded near the city of Carcassonne. In addition to significant considerable material damage, 15 people lost their lives during this flood and 99 people were injured. After the event, the CNRM proposed new forecast ensembles, targeting the possibility of short-term nowcasting (0-6h) of this phenomenon. These ensembles are based on the several NWP models of Météo France: the first ensemble corresponds to the operational AROME-PE product (12 members), the second is a combination of the AROME-PE and AROME-PI models (18 members); finally, the last ensemble corresponds to the second one with a spatial perturbation (90 members). In addition to these ensemble forecasts, ANTILOPE J+1 high resolution observed precipitation data are available.  The work presented here aims to evaluate, from a hydrological point of view, these three rainfall ensembles specifically designed to improve short-range rainfall now casting. Based on the CINECAR distributed hydrological model, discharge ensembles are calculated for nearly 1200 sub-watersheds with an elementary drainage area of 5km². These  forecasts  are compared for each sub-basin with the CINECAR simulation obtained with ANTILOPE J+1 rainfall data.This evaluation approach enables to compensate the lack of discharge observations during the event and to enlarge the dataset used for evaluation. The evaluation results presented combine synthetic scores (CRPS and rank diagrams) often used for ensemble forecasts, but also a user-oriented evaluation framework based on threshold exceedance detection and anticipation. Thresholds for each sub-watershed correspond to the 5, 10, 20 and 50 year return period discharges (SHYREG database). ROC curves are at first established independently of the level of anticipation. In a second time, the anticipation delays are analyzed,. This work finally reveals that (1) synthetic ensemble forecast evaluation scores are not always sufficient to evaluate forecasts; (2) the user oriented evaluation shows a clear hierarchy between the three forecast product ensembles in terms of threshold exceedance detection, but not in terms of anticipation levels.