Ask me anything: Toward open-purpose modeling in hydrology

The recent success of large language models stems in part from the foundation model approach. Foundation models are trained to learn general representations that can be adapted to a range of downstream tasks with little to no retraining. Not having to train a new model from scratch for each task saves resources and accelerates scientific discovery. In this contribution, we present a foundation model approach for probabilistic multivariate geoscientific time series modeling. The proposed neural network architecture learns the joint distribution of multivariate hydrological time series data. This is achieved by training the model to infer subsets of target variables from subsets of predictor variables in an alternating manner. Thereby, the model learns to generate conditional predictions of any involved variable from whatever variables are available. This includes the standard task of predicting discharge from precipitation, but also allows backward inference of variables upstream in the causal pathway. Such anticausal modeling is inherently uncertain. Our approach acknowledges this by its probabilistic variational inference design. We train and evaluate our model on a detailed, heterogeneous, real-world hydrological dataset. We investigate the model's ability to capture dependencies among multiple time series and to accurately reconstruct missing variables with calibrated uncertainty estimates. Furthermore, we compare the performance of our open-purpose model to that of multiple traditional single-purpose models trained for specific inference tasks. Our results suggest that the foundation model approach is feasible in hydrology and allows resource-efficient modeling across diverse inference tasks.