Exploring the Limits of Spatial Generalization Ability in Deep Learning Models for Hydrology

State-of-the-art deep learning models for streamflow prediction (so-called Entity-Aware models, EA) integrate information about physical catchment properties (static features) with climate forcing data (dynamic features) from multiple catchments simultaneously. However, recent studies challenge the notion that this approach truly leverages generalization ability. We explore this issue by conducting experiments running Long-Short Term Memory (LSTM) networks across multiple temporal and spatial in-sample and out-of-sample setups using the CAMELS-US dataset. We compare LSTMs equipped with static features with ablated variants lacking these features. Our findings reveal that the superior performance of EA models is primarily driven by meteorological data, with negligible contributions by static features, particularly in spatial out-of-sample tests. We conclude that EA models cannot generalize to new locations based on provided physical catchment properties. This suggests that current methods of encoding static feature information in our models may need improvement, and that the quality of static features in the hydrologic domain might be limited. We contextualize our results with observations made in the broader deep learning field, which increasingly grapples with the challenges of (lacking) generalization ability in state-of-the-art deep learning models.