Evaluation of Hydro-meteorological Ensemble Forecasts in Small Catchments in Saxony

The application of ensemble forecasting for predicting extreme weather events and floods, necessitates a thorough assessment of its performance and reliability. Consequently, evaluation emerges as a crucial step, offering valuable insights into the overall predictive skill of ensemble forecasts. Notably, limited attention has been given to evaluating the utility of ensemble forecasts as early warning tools in small catchments that are characterized by rapid hydrological processes and flash floods. This study addresses this gap by focusing on the performance analysis of the ensemble weather forecast provided by the Deutscher Wetterdienst (DWD) in ten small catchments distributed across three regions in Saxony, Germany. The findings aim to contribute to a deeper understanding of the effectiveness of ensemble forecasting in the context of early warning systems for small catchments.

To realize this research, a specialized computational tool particularly tailored for the structure and format of the forecast products was developed. It encompasses an arsenal of evaluation metrics, including contingency table-based metrics that examine forecasts from the perspective of extreme events (e.g., false and true alarm rates, accuracy, area under the ROC curve). Additionally, the tool incorporates metrics treating the entire forecast ensemble as a probability distribution, measuring its degree of conformity with the ground truth (e.g., CRPS & Brier score).

The evaluation exercise involved a numerical comparison between modelled forecasts and actual measured observations. For meteorological forecasts, the evaluation was conducted using the numerical weather prediction models ICON-D2-EPS/COSMO-D2-EPS against RADOLAN-RW radar observations. In the case of hydrological forecasts, the modelled runoff ensemble forecasts were compared against gauged catchment runoff measurements.

Upon a comprehensive evaluation of ensemble predictions from various perspectives, including accuracy and reliability, it is observed that the utilization of ensemble forecasting in small catchments demonstrates a satisfactory level of performance. Notably, the central region of Saxony exhibited slightly superior performance compared to the other two test regions. Furthermore, the results indicate a tendency for the ensemble average to consistently overestimate observations in both rainfall and runoff forecasts. It is also equally important to note that this tendency is primarily attributed to the mathematical approach employed in the analysis.

In conclusion, the collective findings of this research offer valuable insights into the practical application of ensemble forecasts for decision-making in each of the small catchments. Despite the overestimation tendency of the forecasts, the overall performance and reliability of ensemble forecasting in small catchments, especially in the central region of Saxony, suggest its potential as a valuable tool for informed decision-making in hydrological and meteorological contexts.