A polyglot tool for the evaluation of deterministic and probabilistic streamflow predictions

Whether they refer to it as validation, verification, or evaluation, hydrological practitioners regularly need to compute performance metrics to measure the differences between observed and simulated/predicted streamflow time series. While the metrics used are often the same (MAE, NSE, KGE, Brier, CRPS, etc.), the tools used to compute them are seldom the same. In some cases, specific tools are not used and the computation of the metrics are directly hand written in the scripts used to analyse model outputs. In addition, the computation of performance metrics is often accompanied with a variety of pre- and post-processing steps that are rarely documented (e.g. handling of missing data, data transformation, selection of events, uncertainty estimation). This can be error prone and hinder the reproducibility of published results. The sharing of tools computing these performance metrics is likely limited by the variety of programming environments in the hydrological community, and by well-established practices in operational environments that are difficult to modify. In order to enable the sharing between researchers and practitioners and move towards more reproducible hydrological science, we argue that an evaluation tool for streamflow predictions must be polyglot (i.e. that it must be usable in several programming languages) and that it must not only compute the performance metrics themselves, but also the pre- and post-processing steps required to compute them. To this end, we present a new open source, polyglot, and compiled tool for the evaluation of deterministic and probabilistic streamflow predictions. The tool, named “evalhyd”, can be used in Python, in R, and as a command line tool. We will present the concept behind its development and illustrate how it works in practice through examples from operational streamflow predictions in France. We will also discuss further steps and remaining challenges in the evaluation of hydrological model predictions.