Assessment of a global hydrological service by application-based metrics

The calibration and validation of hydrological models often involve a suite of established statistical metrics, which may not always match the needs of local stakeholders, thereby constraining the evaluative scope, particularly in the context of global climate services. This study introduces an alternative, complementary evaluation approach by formulating two types of application-based evaluation metrics (Du et al., 2024), representing model performance in terms of (i) temporal matching of the extreme quantiles and (ii) reproduction of the maximized split-sample difference in flow signatures. The introduced metrics are compared to conventional statistical metrics, at seven case study areas across the world, with three model settings representing different datasets and calibrations, generated from the global hydrological model World-Wide HYPE (WWH; Arheimer et al., 2020). The different performances found using application-based and conventional metrics, respectively, reveal their ability to uncover the models' capability in various aspects. Ultimately, the comprehensive analysis of conventional and application-based metrics allows us to delineate two scenarios for model application: generally applicable models, and conditionally applicable models. For example, in some areas the WWH model, when applied with global dataset and local calibration, is well capable of producing predictions for the timing of extreme quantiles and the relative difference in flow signatures, even though it may not excel according to conventional evaluation metrics. Consequently, this model can be classified as conditionally applicable, suitable for areas where local data is scarce, yet providing reliable information that can aid decision-makers in developing strategies for water resources management.

Arheimer, B., Pimentel, R., Isberg, K., Crochemore, L., Andersson, J.C.M., Hasan, A.,  Pineda, L. (2020). Global catchment modelling using World-Wide HYPE (WWH), open data, and stepwise parameter estimation. Hydrology and Earth System Sciences, 24, 535-559.

Du, Y., Olsson, J., Isberg, K., Strömqvist, J., Hundecha., Y., Silva, B.C., Rafee, S.A.A., Fragoso Jr., C.R., Beldring, S., Hansen, A., Uvo, C.B., Sörensen, J. (2024). Application-based evaluation of multi-basin hydrological models. Journal of Hydrology, under revision.