Feature importance for deep learning rainfall-runoff modeling in the boreal zone

Despite deep learning’s recent performance dominance in rainfall-runoff modeling, we still don’t know which variables truly matter in the boreal zone for it. Only a few studies have attempted to determine what variables have largest influence on the prediction quality, and none have done so in the boreal zone. Most feature importance studies have also failed to account for the strong relationships between different covariates present in hydrometeorological datasets and the detrimental effects these pose for many popular feature importance methods. The aim of this study is to address this research gap and increase knowledge on the dominant drivers of rainfall-runoff processes in the boreal zone. More specifically, we sought to create a ranking of feature importances for large selection of catchments and to identify and explain regional differences in the importances.

As a baseline, an ensemble of long short-term memory networks was trained to predict daily runoff for 101 Finnish catchments using 13 dynamic meteorological variables and 36 static attributes from the CAMELS-FI (Catchment Attributes and MEteorology for Large-sample Studies, FInland) dataset. To robustly determine feature importance, three different methods were employed, each involving leaving variables out, retraining the model and evaluating the change in performance, across several performance metrics. The first method was leave-one-covariate-out (LOCO), second was leave-one-covariate-in (LOCI) and third excluded the variable of interest as well as all the correlated variables (leave-one-group-out, LOGO). LOCI was implemented separately to static and dynamic features, such that static features received all dynamic inputs and vice versa.

The results demonstrate significant variations in feature importance both between the different setups and among catchments. The baseline mode performed excellently (mean KGE 0.85). LOCI revealed that snow-related information is more important than precipitation outside the southwest coast of Finland, for multiple metrics related to mean and high flow conditions. This is much further south than previous research has suggested. However, precipitation was the only feature with substantial decline in performance in a LOCO setting (mean KGE 0.74), indicating that it provides information that is both important and unique and that other features are (almost) fully reconstructible from collinear features. Removal of all static attributes reduced the predictive power of the model substantially (mean KGE 0.67). The decline in performance was not spatially uniform. It was greatest in catchments that deviate most from ”average” catchment properties, particularly those with large lake area. The importance of lakes is further supported by the fact that the performance can be mostly restored by reintroducing lake area percentage back to the data (mean KGE 0.79).

This study highlights three key considerations for feature importance analysis in data driven hydrological modeling.

First, focusing solely on global feature importance overlooks regional differences and variables that are important to specific catchments.

Second, hydrologists should account for the correlation structure of hydrological datasets, both when selecting a feature importance method and when interpreting the results.

Third, we argue that the methods examined here measure different aspects of feature importance, and none alone would be sufficient to provide a complete understanding.