HS3.5 | Explainable and hybrid machine learning in hydrology
Explainable and hybrid machine learning in hydrology
Co-organized by ESSI1/NP4
Convener: Shijie JiangECSECS | Co-conveners: Dapeng FengECSECS, Marvin HögeECSECS, Basil KraftECSECS, Lu LiECSECS
Orals
| Mon, 15 Apr, 08:30–12:25 (CEST)
 
Room 2.44
Posters on site
| Attendance Mon, 15 Apr, 16:15–18:00 (CEST) | Display Mon, 15 Apr, 14:00–18:00
 
Hall A
Orals |
Mon, 08:30
Mon, 16:15
The complexity of hydrological systems poses significant challenges to their prediction and understanding capabilities. The rise of machine learning (ML) provides powerful tools for modeling these intricate systems. However, realizing their full potential in this field is not just about algorithms and data, but requires a cooperative interaction between domain knowledge and data-driven power. This session aims to explore the frontier of this convergence, examining how prior understanding of hydrological and land surface processes or causal representations can be incorporated into data-driven models, and conversely, how ML might enrich our causal or physical understanding of system dynamics and mechanisms.

We invite researchers working at the intersection of explainable ML/AI and hydrological or Earth system sciences to share their methods, results, and insights. Submissions are welcome on topics including, but not limited to:

- Explainability and transparency in ML/AI modeling of hydrological and Earth systems;
- Integration of hydrological processes and knowledge in ML/AI models;
- Multiscale and multiphysics representation in ML/AI models;
- Causal representation learning in hydrological and earth systems;
- Strategies for balancing model performance and interpretability;
- Leveraging insights from data science and XAI to deepen physical understanding;
- Data-driven approaches to causal analysis in hydrological and Earth systems;
- Challenges, limitations, and solutions related to hybrid models and XAI.

Orals: Mon, 15 Apr | Room 2.44

Chairpersons: Shijie Jiang, Marvin Höge, Basil Kraft
08:30–08:35
08:35–08:55
|
EGU24-3028
|
HS3.5
|
solicited
|
Highlight
|
On-site presentation
Louise Slater, Gemma Coxon, Manuela Brunner, Hilary McMillan, Le Yu, Yanchen Zheng, Abdou Khouakhi, Simon Moulds, and Wouter Berghuijs

Explaining the spatially variable impacts of flood-generating mechanisms is a longstanding challenge in hydrology, with increasing and decreasing temporal flood trends often found in close regional proximity. Here, we develop a machine learning-informed approach to unravel the drivers of seasonal flood magnitude and explain the spatial variability of their effects in a temperate climate. We employ 11 observed meteorological and land cover time series variables alongside 8 static catchment attributes to model flood magnitude in 1268 catchments across Great Britain over four decades. We then perform a sensitivity analysis to understand how +10% precipitation, +1°C air temperature, or +10 percentage points of urbanisation or afforestation affect flood magnitude in catchments with varying characteristics. Our simulations show that increasing precipitation and urbanisation both tend to amplify flood magnitude significantly more in catchments with high baseflow contribution and low runoff ratio, which tend to have lower values of specific discharge on average. In contrast, rising air temperature (in the absence of changing precipitation) decreases flood magnitudes, with the largest effects in dry catchments with low baseflow index. Afforestation also tends to decrease floods more in catchments with low groundwater contribution, and in dry catchments in the summer. These reported associations are significant at p<0.001. Our approach may be used to further disentangle the joint effects of multiple flood drivers in individual catchments.

How to cite: Slater, L., Coxon, G., Brunner, M., McMillan, H., Yu, L., Zheng, Y., Khouakhi, A., Moulds, S., and Berghuijs, W.: Spatial sensitivity of river flooding to changes in climate and land cover through explainable AI, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-3028, https://doi.org/10.5194/egusphere-egu24-3028, 2024.

08:55–09:05
|
EGU24-4105
|
HS3.5
|
ECS
|
On-site presentation
Shengyu Kang, Jiabo Yin, Louise Slater, Pan Liu, and Dedi Liu

As the planet warms, the frequency and severity of weather-related hazards such as floods are intensifying, posing substantial threats to communities around the globe. Rising flood peaks and volumes can claim lives, damage infrastructure, and compromise access to essential services. However, the physical mechanisms behind global flood evolution are still uncertain, and their implications for socioeconomic systems remain unclear. In this study, we leverage a supervised machine learning technique to identify the dominant factors influencing daily streamflow. We then propose a physics-constrained cascade model chain which assimilates water and heat transport processes to project bivariate risk (i.e. flood peak and volume together), along with its socioeconomic consequences. To achieve this, we drive a hybrid deep learning-hydrological model with bias-corrected outputs from twenty global climate models (GCMs) under four shared socioeconomic pathways (SSPs). Our results project considerable increases in flood risk under the medium to high-end emission scenario (SSP3-7.0) over most catchments of the globe. The median future joint return period decreases from 50 years to around 27.6 years, with 186 trillion dollars and 4 billion people exposed. Downwelling shortwave radiation is identified as the dominant factor driving changes in daily streamflow, accelerating both terrestrial evapotranspiration and snowmelt. As future scenarios project enhanced radiation levels along with an increase in precipitation extremes, a heightened risk of widespread flooding is foreseen. This study aims to provide valuable insights for policymakers developing strategies to mitigate the risks associated with river flooding under climate change.

How to cite: Kang, S., Yin, J., Slater, L., Liu, P., and Liu, D.: Global flood projection and socioeconomic implications under a physics-constrained deep learning framework, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-4105, https://doi.org/10.5194/egusphere-egu24-4105, 2024.

09:05–09:15
|
EGU24-12981
|
HS3.5
|
ECS
|
On-site presentation
Annie Yuan-Yuan Chang, Konrad Bogner, Maria-Helena Ramos, Shaun Harrigan, Daniela I.V. Domeisen, and Massimiliano Zappa

In recent years, the European Alpine space has witnessed unprecedented low-flow conditions and drought events, affecting various economic sectors reliant on sufficient water availability, including hydropower production, navigation and transportation, agriculture, and tourism. As a result, there is an increasing need for decision-makers to have early warnings tailored to local low-flow conditions.

The EU Copernicus Emergency Management Service (CEMS) European Flood Awareness System (EFAS) has been instrumental in providing flood risk assessments across Europe with up to 15 days of lead time since 2012. Expanding its capabilities, the EFAS also generates long-range hydrological outlooks from sub-seasonal to seasonal horizons. Despite its original flood-centric design, previous investigations have revealed EFAS’s potential for simulating low-flow events. Building upon this finding, this study aims to leverage EFAS's anticipation capability to enhance the predictability of drought events in Alpine catchments, while providing support to trans-national operational services.

In this study, we integrate the 46-day extended-range EFAS forecasts into a hybrid setup for 106 catchments in the European Alps. Many studies have demonstrated Long Short-Term Memory (LSTM)’s capacity to produce skillful hydrological forecasts at various time scales. Here we employ the deep learning algorithm Temporal Fusion Transformer (TFT), an algorithm that combines aspects of LSTM networks with the Transformer architecture. The Transformer's attention mechanisms can focus on relevant time steps across longer sequences enabling TFT to capture both local temporal patterns as well as global dependencies. The role of the TFT is to improve the accuracy of low-flow predictions and to understand their spatio-temporal evolution. In addition to EFAS data, we incorporate features such as European weather regime data, streamflow climatology, and hydropower proxies. We also consider catchment characteristic information including glacier coverage and lake proximity. By incorporating its various attention mechanisms, makes TFT a more explainable algorithm than LSTMs, which helps us understand the driving factor for the forecast skill. Our evaluation uses EFAS re-forecast data as the benchmark and measures the reliability of ensemble forecasts using metrics like the Continuous Ranked Probability Skill Score (CRPSS).

Preliminary results show that a hybrid approach using the TFT algorithm can reduce the flashiness of EFAS during drought periods in some catchments, thereby improving drought predictability. Our findings will contribute to evaluating the potential of these forecasts for providing valuable information for skillful early warnings and assist in informing regional and local water resource management efforts in their decision-making.

How to cite: Chang, A. Y.-Y., Bogner, K., Ramos, M.-H., Harrigan, S., Domeisen, D. I. V., and Zappa, M.: Using Temporal Fusion Transformer (TFT) to enhance sub-seasonal drought predictions in the European Alps, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-12981, https://doi.org/10.5194/egusphere-egu24-12981, 2024.

09:15–09:25
|
EGU24-14280
|
HS3.5
|
On-site presentation
Sheng Wang, Rui Zhou, Egor Prikaziuk, Kaiyu Guan, René Gislum, Christiaan van der Tol, Rasmus Fensholt, Klaus Butterbach-Bahl, Andreas Ibrom, and Jørgen Eivind Olesen

Accurately quantifying water and carbon fluxes between terrestrial ecosystems and the atmosphere is highly valuable for understanding ecosystem biogeochemical processes for climate change mitigation and ecosystem management. Remote sensing can provide high spatial and temporal resolution reflectance data of terrestrial ecosystems to support quantifying evapotranspiration (ET) and gross primary productivity (GPP).  Conventional remote sensing-based ET and GPP algorithms are either based on empirical data-driven approaches or process-based models. Empirical data-driven approaches often have high accuracy for cases within the source data domain, but lack the links to a mechanistic understanding of ecosystem processes. Meanwhile, process-based models have high generalizability with incorporating physically based soil-vegetation radiative transfer processes, but usually have lower accuracy. To integrate the strengths of data-driven and process-based approaches, this study developed a radiative transfer process-guided machine learning approach (PGML) to quantify ET and GPP across Europe. Specifically, we used the Soil Canopy Observation, Photochemistry, and Energy fluxes (SCOPE, van der Tol et al. 2009) radiative transfer model to generate synthetic datasets and developed a pre-trained neural network model to quantify ET and GPP. Furthermore, we utilized field measurements from 63 eddy covariance tower sites from 2016 to 2020 across Europe to fine-tune the neural networks with incorporating physical laws into the cost function. Results show that PGML can significantly improve the SCOPE simulations of net radiation (R2 from 0.91 to 0.96), sensible heat fluxes (R2 from 0.43 to 0.77), ET (R2 from 0.61 to 0.78), and GPP (R2 from 0.72 to 0.78) compared to eddy covariance observations. This study highlights the potential of PGML to integrate machine learning and radiative transfer models to improve the accuracy of land surface flux estimates for terrestrial ecosystems.

How to cite: Wang, S., Zhou, R., Prikaziuk, E., Guan, K., Gislum, R., van der Tol, C., Fensholt, R., Butterbach-Bahl, K., Ibrom, A., and Olesen, J. E.: Quantifying Evapotranspiration and Gross Primary Productivity Across Europe Using Radiative Transfer Process-Guided Machine Learning, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-14280, https://doi.org/10.5194/egusphere-egu24-14280, 2024.

09:25–09:35
|
EGU24-13417
|
HS3.5
|
ECS
|
On-site presentation
Manuel Álvarez Chaves, Eduardo Acuña Espinoza, Uwe Ehret, and Anneli Guthke

Hydrological models play a crucial role in understanding and predicting streamflow. Recently, hybrid models, combining both physical principles and data-driven approaches, have emerged as promising tools to extract insights into system functioning and increases in model predictive skill which are beyond traditional models.

However, the study by Acuña Espinoza et al. (2023) has raised the question whether the flexible data-driven component in a hybrid model might "overwrite" the interpretability of its physics-based counterpart. On the example of conceptual hydrological models with dynamic parameters tuned by LSTM networks, they showed that even in a case where the physics-based representation of the hydrological system is chosen to be nonsensical on purpose, the addition of the flexible data-driven component can lead to a well-performing hybrid model. This compensatory behavior highlights the need for a thorough evaluation of physics-based representations in hybrid hydrological models, i.e., hybrid models should be inspected carefully to understand why and how they predict (so well).

In this work, we provide a method to support this inspection: we objectively assess and quantify the contribution of the data-driven component to the overall hybrid model performance. Using information theory and the UNITE toolbox (https://github.com/manuel-alvarez-chaves/unite_toolbox), we measure the entropy of the (hidden) state-space in which the data-driven component of the hybrid model moves. High entropy in this setting means that the LSTM is doing a lot of "compensatory work", and hence alludes to an inadequate representation of the hydrological system in the physics-based component of the hybrid model. By comparing this measure among a set of alternative hybrid models with different physics-based representations, an order in the degree of realism of the considered representations can be established. This is very helpful for model evaluation and improvement as well as system understanding.

To illustrate our findings, we present examples from a synthetic case study where a true model does exist. Subsequently, we validate our approach in the context of regional predictions using CAMELS-GB data. This analysis highlights the importance of using diverse representations within hybrid models to ensure the pursuit of "the right answers for the right reasons". Ultimately, our work seeks to contribute to the advancement of hybrid modeling strategies that yield reliable and physically reasonable insights into hydrological systems.

References

  • Acuña Espinoza, E., Loritz, R., Álvarez Chaves, M., Bäuerle, N., & Ehret, U. (2023). To bucket or not to bucket? analyzing the performance and interpretability of hybrid hydrological models with dynamic parameterization. EGUsphere, 1–22. https://doi.org/10.5194/egusphere-2023-1980

How to cite: Álvarez Chaves, M., Acuña Espinoza, E., Ehret, U., and Guthke, A.: Evaluating physics-based representations of hydrological systems through hybrid models and information theory, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-13417, https://doi.org/10.5194/egusphere-egu24-13417, 2024.

09:35–09:45
|
EGU24-14666
|
HS3.5
|
On-site presentation
Ye Tian, Weili Tan, and Xing Yuan

Deep learning models for streamflow prediction have been widely used but are often considered as "black boxes" due to their lack of interpretability. To address this issue, the field has recently focused on Explainable Artificial Intelligence (XAI) methods to improve the transparency of these models. In this study, we aimed to investigate the influence of precipitation uncertainty on data-driven modeling and elucidate the hydrological significance of deep learning streamflow modeling in both temporal and spatial dimensions by Explainable Artificial Intelligence techniques. To achieve this, an LSTM model for time series prediction and a CNN-LSTM model for fusion spatial-temporal information are proposed. These models are driven by five sets of reanalyzed datasets. The contribution of precipitation before peak flow to runoff simulation is quantified, in order to identify the most important processes in runoff generation for each river basin. In addition, visualization techniques are employed to analyze the relationship between the weights of the convolutional layers in our models and the distribution of precipitation features. By doing so, we aimed to gain insights into the underlying mechanisms of the models' predictions.

The results of our study revealed several key findings. In the high-altitude areas of the Yangtze River's upper reaches, we found that snowmelt runoff, historical precipitation, and recent precipitation were the combined causes for floods. In the middle reach of the Yangtze River, floods were induced by the combined effect of historical and recent precipitation, except for the Ganjiang River, where historical precipitation events played a major role in controlling flood events. Through the visualization of convolutional layers, we discovered that areas with high convolutional layer weights had a greater impact on the model's predictions. We also observed a high similarity between the weight distribution of the convolutional layers and the spatial distribution of multi-year average precipitation in the upper reach river basins. In the middle reach, the weight distribution of the model's convolutional layers showed a strong correlation with the monthly maximum precipitation in the basin. Overall, this study provides valuable insights into the potential of deep learning models for streamflow prediction and enhances our understanding of the impacts of precipitation in the Yangtze River Basin.

How to cite: Tian, Y., Tan, W., and Yuan, X.: Revealing the key factors and uncertainties in data-driven hydrological prediction using Explainable Artificial Intelligence techniques, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-14666, https://doi.org/10.5194/egusphere-egu24-14666, 2024.

09:45–09:55
|
EGU24-11159
|
HS3.5
|
ECS
|
On-site presentation
Felipe Saavedra, Noemi Vergopolan, Andreas Musolff, Ralf Merz, Carolin Winter, and Larisa Tarasova

Nitrate contamination of water bodies is a major concern worldwide, as it poses a risk of eutrophication and biodiversity loss. Nitrate travels from agricultural land to streams through different hydrological pathways, which are abstrusely activated under different hydrological conditions. Certainly, hydrological conditions can alter the connection between different parts of the catchment and streams, in many cases independent of the discharge levels, leading to modifications in transport dynamics, retention, and nitrate removal rates in the catchment. While enhanced nitrate transport can be linked to high levels of hydrological connectivity, little is known about the effects of the spatial patterns of hydrological connectivity on the transport of nutrients at the catchment scale.

In this study, we combined daily stream nitrate concentration and discharge data at the outlet of 15 predominantly agricultural catchments in the United States (191–16,000 km2 area, 3500 km2 median area, and 77% median agriculture coverage) with soil moisture data from  SMAP-Hydroblocks (Vergopolan et al., 2021). SMAP-Hydroblocks is a hyperresolution soil moisture dataset at the top 5 cm of soil column at 30-m spatial resolution and 2-3 days revisit time (2015-2019), and it is derived through a combination of satellite data, land-surface and radiative transfer modeling, machine learning, and in-situ observations.

We configured a deep learning model for each catchment, driven by 2D soil moisture fields and 1D discharge time series, to evaluate the impact of streamflow magnitude and spatial patterns of soil moisture on streamflow nitrate concentration. The model setup comprises two parallel branches. The first branch incorporates a Long Short-term Memory (LSTM) model, the current state-of-the-art for time-series data modeling, utilizing daily discharge as input data. The second branch contains a Convolutional LSTM network (ConvLSTM) that incorporates daily soil moisture series, the fraction of agriculture of each pixel, and the height above the nearest drainage as a measurement of structural hydrological connectivity. Finally, a fully connected neural network combines the outputs of the two branches to predict the time series of nitrate concentration at the catchment outlet.

Preliminary results indicate that the model performs satisfactorily in one-third of the catchments, with Nash-Sutcliffe Efficiency (NSE) values above 0.3 for the test period, which covers the final 25% of the time series, and this is achieved without tuning the hyperparameters. The model failed to simulate nitrate concentrations (resulting in negative NSE values) typically in larger catchments. Using these simulations and explainable AI, we will quantify the importance of different inputs, in particular, we tested the relative importance of soil moisture for simulating nitrate concentrations. While the literature shows most of the predictive power for nitrate comes from streamflow rates, we show how soil moisture fields add value to the prediction and understanding of hydrologic connectivity. Finally, we will fine-tune the model for each catchment and include more predictors to enhance the reliability of model simulations.

How to cite: Saavedra, F., Vergopolan, N., Musolff, A., Merz, R., Winter, C., and Tarasova, L.: Uncovering the impact of hydrological connectivity on nitrate transport at the catchment scale using explainable AI, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-11159, https://doi.org/10.5194/egusphere-egu24-11159, 2024.

09:55–10:05
|
EGU24-7202
|
HS3.5
|
ECS
|
On-site presentation
Feini Huang, Wei Shangguan, and Shijie Jiang

Land-atmosphere coupling (LAC) involves a variety of interactions between the land surface and the atmospheric boundary layer that are critical to are critical to understanding hydrological partitioning and cycling. As climate change continues to affect these interactions, identifying the specific drivers of LAC variability has become increasingly important. However, due to the complexity of the coupling mechanism, a quantitative understanding of the potential drivers is still lacking. Recently, deep learning has been considered as an effective approach to capture nonlinear relationships within the data, which provides a useful window into complex climatic processes. In this study, we will explore the LAC variability under climate change and its potential drivers by using Convolutional Long Short-term Memory (ConvLSTM) together with explainable AI techniques for attribution analysis. Specifically, the variability of the LAC, defined here as a two-legged index, is used as the modeling target, and variables representing meteorological forcing, land use, irrigation, soil properties, gross primary production, ecosystem respiration, and net ecosystem exchange are the inputs. Our analysis covers global land with a spatial resolution of 0.1° × 0.1° every one day during the period 1979–2019. Overall, the study demonstrates how interpretable machine learning would help us understand the complex dynamics of LAC under changing climatic conditions. We expect the results to facilitate the understanding of terrestrial hydroclimate interactions and hopefully provide multiple lines of evidence to support future water management.

How to cite: Huang, F., Shangguan, W., and Jiang, S.: Identifying potential drivers of land-atmosphere coupling variation under climate change by explainable artificial intelligence, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-7202, https://doi.org/10.5194/egusphere-egu24-7202, 2024.

10:05–10:15
|
EGU24-12574
|
HS3.5
|
ECS
|
Highlight
|
On-site presentation
Eduardo Acuna, Ralf Loritz, Manuel Alvarez, Frederik Kratzert, Daniel Klotz, Martin Gauch, Nicole Bauerle, and Uwe Ehret

Hydrological hybrid models have been proposed as an option to combine the enhanced performance of deep learning methods with the interpretability of process-based models. Among the various hybrid methods available, the dynamic parameterization of conceptual models using LSTM networks has shown high potential. 

In this contribution, we extend our previous related work (Acuna Espinoza et al., 2023) by asking the questions: How well can hybrid models predict untrained variables, and how well do they generalize? We address the first question by comparing the internal states of the model against external data, specifically against soil moisture data obtained from ERA5-Land for 60 basins in Great Britain. We show that the process-based layer can reproduce the soil moisture dynamics with a correlation of 0.83, which indicates a good ability of this type of model to predict untrained variables. Moreover, we compare this method against existing alternatives used to extract non-target variables from purely data-driven methods (Lees et al., 2022), and discuss the differences in philosophy, performance, and implementation. Then, we address the second question by evaluating the capacity of such models to predict extreme events. Following the procedure proposed by Frame et al (2022), we train the hybrid models in low-flow regimes and test them in high-flow situations to evaluate the generalization capacity of such models and compare them against results from purely data-driven methods. Both experiments are done using large-sample data from the CAMELS-US and CAMELS-GB dataset.

With these new experiments, we contribute to answering the question of whether hybrid models give an actual advantage over purely data-driven techniques or not.

References

Acuna Espinoza, E., Loritz, R., Alvarez Chaves, M., Bäuerle, N., & Ehret, U.: To bucket or not to bucket? Analyzing the performance and interpretability of hybrid hydrological models with dynamic parameterization. EGUsphere, 1–22. https://doi.org/10.5194/egusphere-2023-1980, 2023.

Frame, J. M. and Kratzert, F. and Klotz, D. and Gauch, M. and Shalev, G. and Gilon, O. and Qualls, L. M. and Gupta, H. V. and Nearing, G. S., :Deep learning rainfall--runoff predictions of extreme events, Hydrology and Earth System Sciences, 26 ,3377-3392, https://doi.org/10.5194/hess-26-3377-2022, 2022

Lees, T., Reece, S., Kratzert, F., Klotz, D., Gauch, M., De Bruijn, J., Kumar Sahu, R., Greve, P., Slater, L., and Dadson, S. J.: Hydrological concept formation inside long short-term memory (LSTM) networks, Hydrology and Earth System Sciences, 26, 3079–3101, https://doi.org/10.5194/hess-26-3079-2022,  2022.

How to cite: Acuna, E., Loritz, R., Alvarez, M., Kratzert, F., Klotz, D., Gauch, M., Bauerle, N., and Ehret, U.: Analyzing the performance and interpretability of hybrid hydrological models, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-12574, https://doi.org/10.5194/egusphere-egu24-12574, 2024.

Coffee break
Chairpersons: Marvin Höge, Basil Kraft, Shijie Jiang
10:45–10:55
|
EGU24-17842
|
HS3.5
|
ECS
|
Highlight
|
On-site presentation
Zavud Baghirov, Basil Kraft, Martin Jung, Marco Körner, and Markus Reichstein

The integration of machine learning (ML) and process based modeling (PB) in so-called hybrid models, also known as differentiable modelling, has recently gained popularity in the geoscientific community (Reichstein et al. 2019; Shen et al. 2023). The approach aims to address limitations in both ML (data adaptive but difficult to interpret and physically inconsistent) and PB (physically consistent and interpretable but biased). It holds significant potential for studying uncertain processes in the global water cycle (Kraft et al. 2022).

In this work, we developed a differentiable/hybrid model of the global hydrological cycle by fusing deep learning with a custom PB model. The model inputs include air temperature, precipitation, net radiation as dynamic forcings, and static features like soil texture as input to a long short-term memory (LSTM) model. The LSTM represents the uncertain and less understood spatio-temporal parameters which are directly used in a conceptual hydrological model. Simultaneously, we use fully connected neural networks (FCNN) to represent the uncertain spatial parameters. In the hydrological model we represent key water fluxes (e.g. transpiration, evapotranspiration (ET), runoff) and storages (snow, soil moisture and groundwater). The model is constrained against the observation-based data, like terrestrial water storage (TWS) anomalies (GRACE), fAPAR (MODIS) and snow water equivalent (GLOBSNOW).

Building upon previous work (Kraft et al. 2022), we improved the representations of key hydrological processes. We now explicitly estimate vegetation state that is directly used to partition ET into transpiration, soil and interception evaporation. We also estimate rooting-zone water storage capacity—a key hydrological parameter that is still highly uncertain. To asses the robustness of the estimated parameters, we quantify equifinality by training multiple models with random weight initialisation in a 10-fold cross validation setup.

The model learns reasonable spatial and spatio-temporal patterns of critical, yet uncertain, hydrological parameters as latent variables. For example, we assess and show that the estimations of global spatial patterns on rooting-zone water storage capacity and transpiration versus ET are plausible. Equifinality quantification indicates that the dynamic patterns of the modelled water storages are robust, while there is a large uncertainty in the mean of soil moisture and TWS.

References

Kraft, Basil, Martin Jung, Marco Körner, Sujan Koirala, and Markus Reichstein. 2022. “Towards Hybrid Modeling of the Global Hydrological Cycle.” Hydrology and Earth System Sciences 26 (6): 1579–1614.

Reichstein, Markus, Gustau Camps-Valls, Bjorn Stevens, Martin Jung, Joachim Denzler, Nuno Carvalhais, et al. 2019. “Deep Learning and Process Understanding for Data-Driven Earth System Science.” Nature 566 (7743): 195–204.

Shen, Chaopeng, Alison P Appling, Pierre Gentine, Toshiyuki Bandai, Hoshin Gupta, Alexandre Tartakovsky, Marco Baity-Jesi, et al. 2023. “Differentiable Modelling to Unify Machine Learning and Physical Models for Geosciences.” Nature Reviews Earth & Environment 4 (8): 552–67.

How to cite: Baghirov, Z., Kraft, B., Jung, M., Körner, M., and Reichstein, M.: Deep learning based differentiable/hybrid modelling of the global hydrological cycle, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-17842, https://doi.org/10.5194/egusphere-egu24-17842, 2024.

10:55–11:05
|
EGU24-6656
|
HS3.5
|
ECS
|
On-site presentation
Luis De La Fuente, Hoshin Gupta, and Laura Condon

Regionalization is an issue that hydrologists have been working on for decades. It is used, for example, when we transfer parameters from one calibrated model to another, or when we identify similarities between gauged to ungauged catchments. However, there is still no unified method that can successfully transfer parameters and identify similarities between different regions while accounting for differences in meteorological forcing, catchment attributes, and hydrological responses.

Machine learning (ML) has shown promising results in the generalization of its results at temporal and spatial scales for streamflow prediction. This suggests that ML models have learned useful regionalization relationships that we could extract. This study explores how the HydroLSTM representation, a modification of traditional Long Short-Term Memory, can learn meaningful relationships between meteorological forcing and catchment attributes. One promising feature of the HydroLSTM representation is that the learned patterns can generate different hydrological responses across the US. These findings indicate that we can learn more about regionalization by studying ML models.

How to cite: De La Fuente, L., Gupta, H., and Condon, L.: Exploring Catchment Regionalization through the Eyes of HydroLSTM, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-6656, https://doi.org/10.5194/egusphere-egu24-6656, 2024.

11:05–11:15
|
EGU24-4768
|
HS3.5
|
ECS
|
On-site presentation
Qingsong Xu, Yilei Shi, Jonathan Bamber, Ye Tuo, Ralf Ludwig, and Xiao Xiang Zhu

Accurate hydrological understanding and water cycle prediction are crucial for addressing scientific and societal challenges associated with the management of water resources, particularly under the dynamic influence of anthropogenic climate change. Existing work predominantly concentrates on the development of machine learning (ML) in this field, yet there is a clear distinction between hydrology and ML as separate paradigms. Here, we introduce physics-aware ML as a transformative approach to overcome the perceived barrier and revolutionize both fields. Specifically, we present a comprehensive review of the physics-aware ML methods, building a structured community (PaML) of existing methodologies that integrate prior physical knowledge or physics-based modeling into ML. We systematically analyze these PaML methodologies with respect to four aspects: physical data-guided ML, physics-informed ML, physics-embedded ML, and physics-aware hybrid learning. PaML facilitates ML-aided hypotheses, accelerating insights from big data and fostering scientific discoveries. We initiate a systematic exploration of hydrology in PaML, including rainfall-runoff and hydrodynamic processes, and highlight the most promising and challenging directions for different objectives and PaML methods. Finally, a new PaML-based hydrology platform, termed HydroPML, is released as a foundation for applications based on hydrological processes [1]. HydroPML presents a range of hydrology applications, including but not limited to rainfall-runoff-inundation modeling, real-time flood forecasting (FloodCast), rainfall-induced landslide forecasting (LandslideCast), and cutting-edge PaML methods, to enhance the explainability and causality of ML and lay the groundwork for the digital water cycle's realization. The HydroPML platform is publicly available at https://hydropml.github.io/.

[1] Xu, Qingsong, et al. "Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology." arXiv preprint arXiv:2310.05227 (2023).

How to cite: Xu, Q., Shi, Y., Bamber, J., Tuo, Y., Ludwig, R., and Zhu, X. X.: HydroPML: Towards Unified Scientific Paradigms for Machine Learning and Process-based Hydrology, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-4768, https://doi.org/10.5194/egusphere-egu24-4768, 2024.

11:15–11:25
|
EGU24-12068
|
HS3.5
|
ECS
|
On-site presentation
Rajeev Shrestha, Bjarte Beil-Myhre, and Bernt Viggo Matheussen

Accurate prediction of streamflow in ungauged basins is a fundamental challenge in hydrology. The lack of hydrological observations and the inherent complexities in ungauged regions hinder accurate predictions, posing significant hurdles for water resource management and forecasting. Over time, efforts have been made to tackle this predicament, primarily utilizing physical hydrological models. However, these models need to be revised due to their reliance on site-specific data and their struggle to capture complex nonlinear relationships. Recent work by Kratzert et al. (2018) suggests that nonlinear regression models such as LSTM neural networks (Hochreiter & Schmidhuber, 1997) may outperform traditional physically based models. The authors demonstrate the application of LSTM models to ungauged prediction problems, noting that information about physical processes might not have been fully utilized in the modeling setup.

In response to these challenges, this research explores a novel approach by introducing a Hybrid Neural Hydrology (HNH) approach by fusing the strengths of physical hydrological models like Statkraft Hydrology Forecasting Toolbox (SHyFT), developed at Statkraft and the Distributed Regression Hydrological Model (DRM), developed by Matheussen at Å Energi with machine learning model, specifically Neural Hydrology, developed by F. Kratzert and team. By combining the information and structural insights of physically based models with the flexibility and adaptability of machine learning models, HNH seeks to leverage the complementary attributes of these methodologies. The combination is achieved by fusing the uncalibrated physical model with an LSTM based model. This hybridization seeks to enhance the model's adaptability and learning capabilities, leveraging available information from various sources to improve predictions in ungauged areas. Furthermore, this research investigates the impact of clustering catchments based on area to improve model performance.

The data used in this research includes dynamic variables such as precipitation, air temperature, wind speed, relative humidity, and observed streamflow obtained from sources such as the internal database at Å Energi, The Norwegian Water Resources and Energy Directorate (NVE), The Norwegian Meteorological Institute (MET), ECMWF (ERA5) and static attributes such as catchment size, mean elevation, forest fraction, lake fraction and reservoir fraction obtained from CORINE Land Cover and Høydedata (www.hoydedata.no).

This study presents HNH as a novel approach that synergistically integrates the structural insights of physical models with the adaptability of machine learning. Preliminary findings indicate promising outcomes from testing in 65 catchments in southern Norway. This suggests that information about physical processes and clustering catchments based on their similarities significantly improves the prediction quality in ungauged regions. This discovery underscores the potential of using hybrid models and clustering techniques to enhance the performance of predictive models in ungauged basins.

How to cite: Shrestha, R., Beil-Myhre, B., and Matheussen, B. V.: Hybrid Neural Hydrology: Integrating Physical and Machine Learning Models for Enhanced Predictions in Ungauged Basins, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-12068, https://doi.org/10.5194/egusphere-egu24-12068, 2024.

11:25–11:35
|
EGU24-2850
|
HS3.5
|
ECS
|
On-site presentation
Liangjin Zhong, Huimin Lei, and JIngjing Yang

Climate change has exacerbated water stress and water-related disasters, necessitating more precise runoff simulations. However, in the majority of global regions, a deficiency of runoff data constitutes a significant constraint on modeling endeavors. Traditional distributed hydrological models and regionalization approaches have shown suboptimal performance. While current data-driven models trained on large datasets excel in spatial extrapolation, the direct applicability of these models in certain regions with unique hydrological processes may be challenging due to the limited representativeness within the training dataset. Furthermore, transfer learning deep learning models pre-trained on large datasets still necessitate local data for retraining, thereby constraining their applicability. To address these challenges, we present a physics-informed deep learning model based on a distributed framework. It involves spatial discretization and the establishment of differentiable hydrological models for discrete sub-basins, coupled with a differentiable Muskingum method for channel routing. By introducing upstream-downstream relationships, model errors in sub-basins propagate through the river network to the watershed outlet, enabling the optimization using limited downstream runoff data, thereby achieving spatial simulation of ungauged internal sub-basins. The model, when trained solely on the downstream-most station, outperforms the distributed hydrological model in runoff simulation at both the training station and upstream stations, as well as evapotranspiration spatial patterns. Compared to transfer learning, our model requires less training data, yet achieves higher precision in simulating runoff on spatially hold-out stations and provides more accurate estimates of spatial evapotranspiration. Consequently, this model offers a novel approach to hydrological simulation in data-scarce regions with unique processes.

How to cite: Zhong, L., Lei, H., and Yang, J.: Development of a Distributed Physics-informed Deep Learning Hydrological Model for Data-scarce Regions, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-2850, https://doi.org/10.5194/egusphere-egu24-2850, 2024.

11:35–11:55
|
EGU24-262
|
HS3.5
|
solicited
|
Virtual presentation
Chaopeng Shen, Yalan Song, Farshid Rahmani, Tadd Bindas, Doaa Aboelyazeed, Kamlesh Sawadekar, Martyn Clark, and Wouter Knoben

Process-based modeling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. A recently proposed genre of physics-informed machine learning, called “differentiable” modeling (DM, https://t.co/qyuAzYPA6Y), trains neural networks (NNs) with process-based equations (priors) together in one stage (so-called “end-to-end”) to benefit from the best of both NNs and process-based paradigms. The NNs do not need target variables for training but can be indirectly supervised by observations matching the outputs of the combined model, and differentiability critically supports learning from big data. We propose that differentiable models are especially suitable as global hydrologic models because they can harvest information from big earth observations to produce state-of-the-art predictions (https://mhpi.github.io/benchmarks/), enable physical interpretation naturally, extrapolate well (due to physical constraints) in space and time, enforce known physical laws and sensitivities, and leverage progress in modern AI computing architecture and infrastructure. Differentiable models can also synergize with existing global hydrologic models (GHMs) and learn from the lessons of the community. Differentiable GHMs to answer pressing societal questions on water resources availability, climate change impact assessment, water management, and disaster risk mitigation, among others. We demonstrate the power of differentiable modeling using computational examples in rainfall-runoff modeling, river routing, forcing fusion, as well applications in water-related domains such as ecosystem modeling and water quality modeling. We discuss how to address potential challenges such as implementing gradient tracking for implicit numerical schemes and addressing process tradeoffs. Furthermore, we show how differentiable modeling can enable us to ask fundamental questions in hydrologic sciences and get robust answers from big global data.

How to cite: Shen, C., Song, Y., Rahmani, F., Bindas, T., Aboelyazeed, D., Sawadekar, K., Clark, M., and Knoben, W.: Differentiable modeling for global water resources under global change, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-262, https://doi.org/10.5194/egusphere-egu24-262, 2024.

11:55–12:05
|
EGU24-16235
|
HS3.5
|
ECS
|
On-site presentation
Phillip Aarestrup, Jonas Wied Pedersen, Michael Brian Butts, Peter Bauer-Gottwein, and Roland Löwe

Simulations of river flows and water levels are crucial for flood predictions and water resources management. Water levels are easy to observe using sensors, while the mapping between water levels and flows in rivers is usually derived from rating curves. However, rating curves frequently do not include geometry, backwater effects, and/or seasonal variations, which can limit their applicability – especially in stream systems that are affected by seasonal vegetation and backwater effects. To address this, we propose a differentiable model that merges a neural network with a physically based, steady-state implementation of the Saint-Venant equations. 

In the setup, the neural network is trained to predict seasonal variations caused by vegetation growth in Manning’s roughness based on inputs of meteorological forcing and time, while the physical model is responsible for converting flow estimates into water levels along the river channel. The framework efficiently estimates model parameters by tracking gradients through both the physical model and the neural network via backpropagation. This allows us to calibrate parameters for both the runoff and the Manning’s roughness from measured water levels, thus overcoming rating curve limitations while accounting for backwater, river geometry, and seasonal variations in roughness. 

We tested the model on a 20 km stretch of the Vejle River, Denmark, which is both heavily vegetated and affected by backwater from the coast. The model was trained across five water level sensors using two years of data (2020-2022). When evaluated against 10 years of observed flow measurements (2007-2017), the model demonstrated a Mean Absolute Relative Error (MARE) of 10% compared to manually gauged discharge observations. This is comparable to the estimated uncertainty of 10% in the discharge measurements.  

The framework enables a calibration of dynamic Manning roughness within a few hours, and therefore offers a scalable solution for estimating river flows from water levels when cross-section information is available. Potential applications span across many disciplines in water resource management. 

How to cite: Aarestrup, P., Pedersen, J. W., Butts, M. B., Bauer-Gottwein, P., and Löwe, R.: Flow estimation from observed water levels using differentiable modeling for low-lying rivers affected by vegetation and backwater, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-16235, https://doi.org/10.5194/egusphere-egu24-16235, 2024.

12:05–12:15
|
EGU24-20602
|
HS3.5
|
ECS
|
Virtual presentation
Tadd Bindas, Yalan Song, Jeremy Rapp, Kathryn Lawson, and Chaopeng Shen

Recent advancements in flow routing models have enabled learning from big data using differentiable modeling techniques. However, their application remains constrained to smaller basins due to limitations in computational memory and hydrofabric scaling. We propose a novel methodology to scale differentiable river routing from watershed (HUC10) to continental scales using the δMC-CONUS-hydroDL2 model. Mimicking the Muskingum-Cunge routing model, this approach aims to enhance flood wave timing prediction and Manning’s n parameter learning across extensive areas. We employ the δHBV-HydroDL model, trained on the 3000 GAGES-II dataset, for streamflow predictions across CONUS HUC10 basins. These predictions are then integrated with MERIT basin data and processed through our differentiable routing model, which learns reach-scale parameters like Manning’s n and spatial channel coefficient q via an embedded neural network. This approach enhances national-scale flood simulations by leveraging a learned Manning’s n parameterization, directly contributing to the refinement of CONUS-scale flood modeling. Furthermore, this method shows promise for global application, contingent upon the availability of streamflow predictions and MERIT basin data. Our methodology represents a significant step forward in the spatial scaling of differentiable river routing models, paving the way for more accurate and extensive flood simulation capabilities.

How to cite: Bindas, T., Song, Y., Rapp, J., Lawson, K., and Shen, C.: Enhanced Continental Runoff Prediction through Differentiable Muskingum-Cunge Routing (δMC-CONUS-hydroDL2), EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-20602, https://doi.org/10.5194/egusphere-egu24-20602, 2024.

12:15–12:25
|
EGU24-11778
|
HS3.5
|
ECS
|
On-site presentation
Bjarte Beil-Myhre, Bernt Viggo Matheussen, and Rajeev Shrestha

Hydrological modeling has undergone a transformative decade, primarily catalyzed by the groundbreaking data-driven approach introduced by F. Kratzert et al. (2018) utilizing LSTM networks (Hochreiter & Schmidhuber, 1997). These networks leverage extensive datasets and intricate model structures, outshining traditional hydrological models, albeit with the caveat of being computationally intensive during training. This prompts a critical inquiry into the requisite volume and complexity of data for constructing a dependable and resilient hydrological model.


In this study, we employ a hybrid model that amalgamates the strengths of classical hydrological models with the data-driven approach. These modified models are derived from the LSTM models developed by F. Kratzert and team, in conjunction with classical hydrological models such as the Statkraft Hydrology Forecasting Toolbox (SHyFT) from Statkraft and the Distributed Regression Hydrological Model (DRM) by Matheussen at Å Energi. The models were applied to sixty-five catchments in southern Norway, each characterized by diverse features and data records. Our analysis assesses the performance of these models under various scenarios of data availability, considering factors such as:


- Varying numbers of catchments selected based on size or location.
- The duration of the data records utilized for model calibration.
- Specific catchment characteristics and outputs from classical models employed as inputs 
(e.g., area, latitude, longitude, or additional variables).


Preliminary findings indicate that model inputs can be significantly stripped down without compromising model performance. With a limited set of catchment characteristics, the performance approaches that of the model with all characteristics, mitigating added uncertainty and model complexity. Additionally, increasing the length of data records enhances model performance, albeit with diminishing returns. Furthermore, our study reveals that augmenting catchments in the model does not necessarily yield a commensurate improvement in overall model performance. These insights contribute to refining our understanding of the interplay between data, model complexity, and performance in hydrological modeling.


The novelty in this research is that the hybrid models can be applied in a relatively small area, with few catchments and a limited number of climate stations and catchment characteristics compared to the CAMELS setup, used by Kratzert and still achieve improved results. 

How to cite: Beil-Myhre, B., Matheussen, B. V., and Shrestha, R.: How much data is needed for hydrological modeling? , EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-11778, https://doi.org/10.5194/egusphere-egu24-11778, 2024.

Posters on site: Mon, 15 Apr, 16:15–18:00 | Hall A

Display time: Mon, 15 Apr 14:00–Mon, 15 Apr 18:00
Chairpersons: Basil Kraft, Marvin Höge, Shijie Jiang
A.40
|
EGU24-9980
|
HS3.5
|
ECS
Kai Ma and Daming He

In facing the challenges of limited observational streamflow data and climate change, accurate streamflow prediction and flood management in large-scale catchments become essential. This study introducing a time-lag informed deep learning framework to enhance streamflow simulation and flood forecasting. Using the Dulong-Irrawaddy River Basin (DIRB), a less-explored transboundary basin shared by Myanmar, China, and India, as a case study, we have identified peak flow lag days and relative flow scale. Integrating these with historical flow data, we developed an optimal model. The framework, informed by data from the upstream Hkamti sub-basin, significantly outperformed standard LSTM, achieving a Kling-Gupta Efficiency (KGE) of 0.891 and a Nash-Sutcliffe efficiency coefficient (NSE) of 0.904. Notably, the H_PFL model provides a valuable 15-day lead time for flood forecasting, enhancing emergency response preparations. The transfer learning model, incorporating meteorological inputs and catchment features, achieved an average NSE of 0.872 for streamflow prediction, surpassing the process-based model MIKE SHE's 0.655. We further analyzed the sensitivities of the deep learning model and process-based model to changes in meteorological inputs using different methods. Deep learning models exhibit complex sensitivities to these inputs, more accurately capturing non-linear relationships among multiple variables than the process-based model. Integrated Gradients (IG) analysis further demonstrates deep learning model's ability to discern spatial heterogeneity in upstream and downstream sub-basins and its adeptness in characterizing different flow regimes. This study underscores the potential of deep learning in enhancing the understanding of hydrological processes in large-scale catchments and highlights its value for water resource management in transboundary basins under data scarcity.

How to cite: Ma, K. and He, D.: Streamflow Prediction and Flood Forecasting with Time-Lag Informed Deep Learning framework in Large Transboundary Catchments, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-9980, https://doi.org/10.5194/egusphere-egu24-9980, 2024.

A.41
|
EGU24-4325
|
HS3.5
|
ECS
|
Liangkun Deng, Xiang Zhang, and Louise Slater

Hybrid models have shown impressive performance for streamflow simulation, offering better accuracy than process-based hydrological models (PBMs) and superior interpretability than deep learning models (DLMs). A recent paradigm for streamflow modeling, integrating DLMs and PBMs within a differentiable framework, presents considerable potential to match the performance of DLMs while simultaneously generating untrained variables that describe the entire water cycle. However, the potential of this framework has mostly been verified in small and unregulated headwater basins and has not been explored in large and highly regulated basins. Human activities, such as reservoir operations and water transfer projects, have greatly changed natural hydrological regimes. Given the limited access to operational water management records, PBMs generally fail to achieve satisfactory performance and DLMs are challenging to train directly. This study proposes a coupled hybrid framework to address these problems. This framework is based on a distributed PBM, the Xin'anjiang (XAJ) model, and adopts embedded deep learning neural networks to learn the physical parameters and replace the modules of the XAJ model reflecting human influences through a differentiable structure. Streamflow observations alone are used as training targets, eliminating the need for operational records to supervise the training process. The Hanjiang River basin (HRB), one of the largest subbasins of the Yangtze River basin, disturbed by large reservoirs and national water transfer projects, is selected to test the effectiveness of the framework. The results show that the hybrid framework can learn the best parameter sets of the XAJ model depicting natural and human influences to improve streamflow simulation. It performs better than a standalone XAJ model and achieves similar performance to a standalone LSTM model. This framework sheds new light on assimilating human influences to improve simulation performance in disturbed river basins with limited operational records.

How to cite: Deng, L., Zhang, X., and Slater, L.: Towards learning human influences in a highly regulated basin using a hybrid DL-process based framework, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-4325, https://doi.org/10.5194/egusphere-egu24-4325, 2024.

A.42
|
EGU24-7950
|
HS3.5
|
ECS
wang jiao and zhang yongqiang

Predicting streamflow is key for water resource planning, flood and drought risk assessment, and pollution mitigation at regional, national, and global scales. There is a long-standing history of developing physically or conceptually catchment rainfall-runoff models that have been continuously refined over time to include more physical processes and enhance their spatial resolution. On the other hand, machine learning methods, particularly neural networks, have demonstrated exceptional accuracy and extrapolation capabilities in time-series prediction. Both approaches exhibit their strengths and limitations. This leads to a research question: how to effectively balance model complexity and physical interpretability while maintaining a certain level of predictive accuracy. This study aims to effectively combine a conceptual hydrological model, HBV, with machine learning (Transformer, Long Short-Term Memory (LSTM)) using a differentiable modeling framework strategy, tailored to predicting streamflow under diverse climatic and geographical conditions across China. Utilizing the Transformer to optimize and replace certain parameterization processes in the HBV model, a deep integration of neural networks and the HBV model is achieved. This integration not only captures the non-linear relationships that traditional hydrological models struggle to express, but also maintains the physical interpretability of the model. Preliminary application results show that the proposed framework outperforms traditional HBV model and pure LSTM model in streamflow prediction across 68 catchments in China. Based on the test results from different catchments, we have adjusted and optimized the model structure or parameters to better adapt to the unique hydrological processes of each catchment. The application of self-attention mechanisms and a differentiable programming framework significantly enhances the model's ability to capture spatiotemporal dynamics. It is likely that the proposed framework can be widely used for streamflow prediction somewhere else.

How to cite: jiao, W. and yongqiang, Z.: Improving streamflow prediction across China by hydrological modelling together with machine learning, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-7950, https://doi.org/10.5194/egusphere-egu24-7950, 2024.

A.43
|
EGU24-2211
|
HS3.5
|
ECS
|
Milad Panahi, Giovanni Porta, Monica Riva, and Alberto Guadagnini

Addressing the complexities of groundwater modeling, especially under the veil of uncertain physical parameters and limited observational data, poses significant challenges. This study introduces an approach using Physics-Informed Neural Network (PINN) framework to unravel these uncertainties. Termed PINN under uncertainty, PINN-UU, adeptly integrates uncertain parameters within spatio-temporal domains, focusing on hydrological systems. This approach, exclusively built on underlying physical equations, leverages a staged training methodology, effectively navigating high-dimensional solution spaces. We demonstrate our approach through application of reactive transport modeling in porous media, a problem setting relevant to contaminant transport in soil and groundwater. PINN-UU shows promising capabilities in enhancing model reliability and efficiency, and in conducting sensitivity analysis. Our approach is designed to be accessible and engaging, offering insightful contributions to environmental engineering, and hydrological modeling. It represents a step toward deciphering complex geohydrological systems, with broad implications for resource management and environmental science.

How to cite: Panahi, M., Porta, G., Riva, M., and Guadagnini, A.: Staged Learning in Physics-Informed Neural Networks to Model Contaminant Transport under Parametric Uncertainty, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-2211, https://doi.org/10.5194/egusphere-egu24-2211, 2024.

A.44
|
EGU24-6378
|
HS3.5
|
ECS
Iacopo F. Ferrario, Mariapina Castelli, Alasawedah M. Hussein, Usman M. Liaqat, Albrecht Weerts, and Alexander Jacob

The Alpine region is often called the Water Tower of Europe, alluding to its water richness and its function of supplying water through several important European rivers flowing well beyond its geographical boundaries. Climate change projections show that the region will likely experience rising temperatures and changes in precipitation type, frequency, and intensity, with consequences on the spatiotemporal pattern of water availability. Seasonal forecasts could supply timely information for planning water allocation a few months in advance, reducing potential conflicts under conditions of scarce water resources. The overall goal of this study is to improve the seasonal forecasts of hydrological droughts over the entire Alpine region at a spatial resolution (~1 km) that matches the information need by local water agencies, e.g., resolving headwaters and small valleys. In this study we present the progress on the following key objectives:

  • Improving the estimation of distributed model (Wflow_sbm) parameters by finding the optimal transfer function from geophysical attributes to model parameters and upscaling the information to model resolution.
  • Combining physical-hydrological knowledge with data-driven (ML/DL) techniques for improving accuracy and computational performance, without compromising on interpretation
  • Integrating EO-based hydrological fluxes, like streamflow, surface soil moisture, actual evapotranspiration, and snow waters equivalent, with the aim of regularizing the calibration/training, tackling the problem of model parameters equifinality.

Our work is part of the InterTwin project that aims at developing a multi-domain Digital Twin blueprint architecture and implementation platform. We build on the technological solutions developed in InterTwin (e.g. openEO, CWL and STAC) and fully embrace its inspiring principles of open science, reproducibility, and interoperability of data and methods.

How to cite: Ferrario, I. F., Castelli, M., Hussein, A. M., Liaqat, U. M., Weerts, A., and Jacob, A.: Seasonal forecasts of hydrological droughts over the Alps: advancing hybrid modelling applications, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-6378, https://doi.org/10.5194/egusphere-egu24-6378, 2024.

A.45
|
EGU24-4238
|
HS3.5
|
ECS
John Quilty and Mohammad Sina Jahangir

Input variable selection (IVS) is an integral part of building data-driven models for hydrological applications. Carefully chosen input variables enable data-driven models to discern relevant patterns and relationships within data, improving their predictive accuracy. Moreover, the optimal choice of input variables can enhance the computational efficiency of data-driven models, reduce overfitting, and contribute to a more interpretable and parsimonious model. Meanwhile, including irrelevant and/or redundant input variables can introduce noise to the model and hinder its generalization ability.

Three probabilistic IVS methods, namely Edgeworth approximation-based conditional mutual information (EA), double-layer extreme learning machine (DLELM), and gradient mapping (GM), were used for IVS and then coupled with a long short-term memory (LSTM)-based probabilistic deep learning model for daily streamflow prediction. While the EA method is an effective IVS method, DLELM and GM are examples of probabilistic neural network-based IVS methods that have not yet been explored for hydrological prediction. DLELM selects input variables through sparse Bayesian learning, pruning both input and output layer weights of a committee of neural networks. GM is based on saliency mapping, an explainable AI technique commonly used in computer vision that can be coupled with probabilistic neural networks. Both DLELM and GM involve randomization during parameter initialization and/or training thereby introducing stochasticity into the IVS procedure, which has been shown to improve the predictive performance of data-driven models.

The IVS methods were coupled with a LSTM-based probabilistic deep learning model and applied to a streamflow prediction case study using 420 basins spread across the continental United States. The dataset includes 37 candidate input variables derived from the daily-averaged ERA-5 reanalysis data.

Comparing the most frequently selected input variables by EA, DLELM, and GM across the 420 basins revealed that all three models select a similar number of input variables. For example, the top 15 input variables selected by all methods included nine variables that were similar.

The input variables selected by EA, DLELM, and GM were then used in the LSTM-based probabilistic deep learning models for streamflow prediction across the 420 basins. The probabilistic deep learning models were developed and optimized using the top 10 variables selected by each IVS method. The results were compared to a benchmark scenario that used all 37 ERA-5 variables in the prediction model. Overall, the findings show that the GM method results in higher prediction accuracy (Kling-Gupta efficiency; KGE) compared to the other two IVS methods. A median KGE of 0.63 was obtained for GM, whereas for the EA, DLELM, and all input variables’ scenario, KGE scores of 0.61, 0.60, and 0.62 were obtained, respectively.

DLELM and GM are two AI-based techniques that introduce elements of interpretability and stochasticity to the IVS process. The results of the current study are expected to contribute to the evolving landscape of data-driven hydrological modeling by introducing hitherto unexplored neural network-based IVS to pursue more parsimonious, efficient, and interpretable probabilistic deep learning models.

How to cite: Quilty, J. and Jahangir, M. S.: Letting neural networks talk: exploring two probabilistic neural network models for input variable selection, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-4238, https://doi.org/10.5194/egusphere-egu24-4238, 2024.

A.46
|
EGU24-9153
|
HS3.5
Lijun Wang, Liangsheng Shi, Wenxiang Song, Leilei He, Yanling Wang, and Shijie Jiang

Due to its fundamental role in the Earth's hydrological system, understanding and predicting soil water processes has long been important. Rapidly evolving machine learning methods have shown remarkable predictive capabilities, but due to a lack of process understanding, are still insufficient when data is suboptimal or the environment varies. Efforts to integrate prior physical knowledge into learning machines have proven effective in improving the robustness of machine learning models in a complex environment. This improvement has relied heavily on domain-specific knowledge, but the potential for machine learning itself to autonomously understand complex environmental processes, such as soil water movement, is still an intriguing possibility.

To explore this possibility, we have developed a novel causal deep learning method specifically designed to capture the underlying causal mechanism of soil hydrodynamics, thereby enabling the model to adaptively predict vertical profiles of soil moisture in changing environments. By using time-series data from heterogeneous environments, our model is trained to understand the causal interplay between meteorological forcing, environmental factors, and various high-level physical processes that govern the spatiotemporal distribution of soil moisture. The key innovation of our method lies in its ability to use the learned causal relationships as a guide. Even when faced with sparse data in new and untested environments, the model can estimate the differences and adapt its predictions accordingly. Our method is expected to not only enhance soil moisture predictions in new environments, but also provide potentially new insights into a deeper synergy between physical knowledge and data-driven learning.

How to cite: Wang, L., Shi, L., Song, W., He, L., Wang, Y., and Jiang, S.: Deep causal learning for soil water dynamics: Exploring latent causality and improving predictive adaptability, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-9153, https://doi.org/10.5194/egusphere-egu24-9153, 2024.

A.47
|
EGU24-9319
|
HS3.5
|
ECS
|
Bu Li, Ting Sun, Fuqiang Tian, and Guangheng Ni

Large alpine basins on the Tibetan Plateau (TP) provide abundant water resources crucial for hydropower generation, irrigation, and daily life. In recent decades, the TP has been significantly affected by climate change, making it crucial to understand the runoff response to climate change are essential for water resources management. While limited knowledge of specific alpine hydrological processes has constrained the accuracy of hydrological models and heightened uncertainties in climate change assessments. Recently, hybrid hydrological models have come to the forefront, synergizing the exceptional learning capacity of deep learning with a rigorous adherence to hydrological knowledge of process-based models. These models exhibit considerable promise in achieving precision in hydrological simulations and conducting climate change assessments. However, a notable limitation of existing hybrid models lies in their failure to incorporate spatial information and describe alpine hydrological processes, which restricts their applicability in hydrological modeling and climate change assessment in large alpine basins. To address this issue, we develop a set of hybrid distributed hydrological models by employing a distributed process-based model as the backbone, and utilizing embedded neural networks (ENNs) to parameterize and replace different internal modules. The proposed models are tested on three large alpine basins on the Tibetan Plateau. Results are compared to those obtained from hybrid lumped models, state-of-the-art distributed hydrological model, and DL models. A climate perturbation method is further used to evaluate the alpine basins' runoff response to climate change.Results indicate that proposed hybrid hydrological models can perform well in predicting runoff in large alpine basins. The optimal hybrid model with Nash-Sutcliffe efficiency coefficients (NSEs) higher than 0.87 shows comparable performance to state-of-the-art DL models. The hybrid distributed model also exhibits remarkable capability in simulating hydrological processes at ungauged sites within the basin, markedly surpassing traditional distributed models. Besides, runoff exhibits an amplification effect in response to precipitation changes, with a 10% precipitation change resulting in a 15–20% runoff change in large alpine basins. An increase in temperature enhances evaporation capacity and changes the redistribution of rainfall and snowfall and the timing of snowmelt, leading to a decrease in the total runoff and a reduction in the intra-annual variability of runoff. Overall, this study provides a high-performance tool enriched with explicit hydrological knowledge for hydrological prediction and improves our understanding about runoff’s response to climate change in large alpine basins on the TP. 

How to cite: Li, B., Sun, T., Tian, F., and Ni, G.: Developing hybrid distributed models for hydrological simulation and climate change assessment in large alpine basins, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-9319, https://doi.org/10.5194/egusphere-egu24-9319, 2024.

A.48
|
EGU24-5433
|
HS3.5
|
ECS
Experimental and numerical techniques to predict the response of complex and ecologically sensitive fen wetlands to shifts in climate and land use changes
(withdrawn)
Lucía Magnano, Nico Heitepriem, Roland Baatz, and Irina Engelhardt
A.49
|
EGU24-6965
|
HS3.5
|
ECS
Han Zhang, Lu Li, and Yongjiu Dai

Accurate representation of snow cover fraction (SCF) is vital for terrestrial simulation, as it significantly affects surface albedo and land surface radiation. In land models, SCF is parameterized using snow water equivalent and snow depth. This study introduces a novel machine learning-based parameterization, which incorporates the light-GBM regression algorithm and additional input features: surface air temperature, humidity, leaf area index, and the standard deviation of topography. The regression model is trained with input features from the Common Land Model (CoLM) simulations and the labels from the Moderate Resolution Imaging Spectroradiometer (MODIS) observations on a daily scale. Offline verification indicates significant improvements for the new scheme over multiple traditional parameterizations.

Moreover, this machine learning-based parameterization has been online coupled with the CoLM using the Message Passing Interface (MPI). In online simulations, it substantially outperforms the widely used Niu and Yang (2007) scheme, improving the root mean square errors and temporal correlations of SCF on 80% of global grids. Additionally, associated land surface temperature and hydrological processes also benefit from the enhanced estimation of SCF. The new solution also shows good portability as it also demonstrates similar enhancements when it is directly used in a global 1° simulation, even though it was trained at a 0.1° resolution.

How to cite: Zhang, H., Li, L., and Dai, Y.: A Machine Learning Based Snow Cover Parameterization  for Common Land Model (CoLM) , EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-6965, https://doi.org/10.5194/egusphere-egu24-6965, 2024.

A.50
|
EGU24-9281
|
HS3.5
Inferring soil water movement from soil moisture distribution with deep learning method
(withdrawn)
Sheng Ye, Jiyu Li, and Qihua Ran
A.51
|
EGU24-20112
|
HS3.5
|
ECS
Boen Zhang, Louise Slater, Simon Moulds, Michel Wortmann, Yinxue Liu, Jiabo Yin, and Xihui Gu

Reliable flood projection is crucial for designing suitable flood protection structures and for enhancing resilience in vulnerable regions. However, projections of future flooding suffer from cascading uncertainties arising from the climate model outputs, emission scenarios, hydrological models, and the shortage of observations in data-sparse regions. To overcome these limitations, we design a new hybrid model, blending machine learning and climate model simulations, for global-scale projection of river flooding. This is achieved by training a random forest model directly on climate simulations from 20 CMIP6 models over the historical period (1985−2014), with extreme discharges observed at approximately 15,000 hydrologic stations as the target variable. The random forest model also includes static geographic predictors including land cover, climate, geomorphology, soil, human impacts, and hydrologic signatures. We make the explicit assumption that the random forest model can ‘learn’ systematic biases in the relationship between the climate simulations and flood regimes in different regions of the globe. We then apply the well-calibrated random forest model to a new vector-based, global river network in approximately 18.51 million reaches with drainage areas greater than 100 km2. Global changes in flood hazard are projected for the 21st century (2015−2100) under SSP2-4.5 and SSP5-8.5. We show that the data-driven method reproduces historical annual maximum discharges better than the physically-based hydrological models driven by bias-corrected climate simulations in the ISIMIP3b experiment. We then use the machine learning model with explainable AI to diagnose spatial biases in the climate simulations and future flood projections in different regions of the globe.

How to cite: Zhang, B., Slater, L., Moulds, S., Wortmann, M., Liu, Y., Yin, J., and Gu, X.: Data-driven global projection of future flooding in 18.5 million river reaches, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-20112, https://doi.org/10.5194/egusphere-egu24-20112, 2024.

A.52
|
EGU24-451
|
HS3.5
|
ECS
|
Meera Mohan and Nagesh Kumar D

Streamflow can be affected by numerous factors, such as solar radiation, underlying surface conditions, and atmospheric circulation which results in nonlinearity, uncertainty, and randomness in streamflow time series. Diverse conventional and Deep Learning (DL) models have been applied to recognize the complex patterns and discover nonlinear relationships in the hydrological time series and incorporating multi-variables in deep learning can match or improve streamflow forecasts and hopes to improve extreme value predictions. Multivariate approaches surpass univariate ones by including additional time series as explanatory variables. Deep neural networks (DNNs) excel in multi-horizon time series forecasting, outperforming classical models. However, determining the relative contribution of each variable in streamflow remains challenging due to the black-box nature of DL models.

 

We propose utilizing the advanced Temporal Fusion Transformers (TFT) deep-learning technique to model streamflow values across various temporal scales, incorporating multiple variables. TFT's attention-based architecture enables high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. Additionally, the model identifies the significance of each input variable, recognizes persistent temporal patterns, and highlights extreme events. Despite its application in a few studies across different domains, the full potential of this model remains largely unexplored. The study focused on Sundargarh, an upper catchment of the Mahanadi basin in India, aiming to capture pristine flow conditions. QGIS was employed to delineate the catchment, and daily streamflow data from 1982 to 2020 were obtained from the Central Water Commission. Input variables included precipitation, potential evaporation, temperature, and soil water volume at different depths. Precipitation and temperature datasets were obtained from India Meteorological Department (IMD) datasets, while other variables were sourced from the ECMWF fifth-generation reanalysis (ERA-5). Hyperparameter tuning was conducted using the Optuna optimization framework, known for its efficiency and easy parallelization. The model trained using quantile loss function with different combinations of quantiles, demonstrated superior performance with upper quantiles. Evaluations using R2 and NSE indicated good performance in monthly streamflow predictions for testing sets, particularly in confidently predicting low and medium flows. While peak flows were well predicted at certain timesteps, there were instances of underperformance. Unlike other ML algorithms, TFT can learn seasonality and lag analysis patterns directly from raw training data, including the identification of crucial variables. The model underwent training for different time periods, checking for performance improvement with increased length of data. To gain a better understanding of how distinct sub-processes affect streamflow patterns at various time scales, the model was applied at pentad and daily scales. Evaluation at extreme values prompted an investigation into improving predictions through quantile loss function adjustments. Given the computational expense of daily streamflow forecasting using TFT with multiple variables, parallel computing is employed. Results demonstrated considerable accuracy, but validating TFT's interpretive abilities require testing alternative ML models.

 

How to cite: Mohan, M. and Kumar D, N.: Multivariate multi-horizon streamflow forecasting for extremes and their interpretation using an explainable deep learning architecture, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-451, https://doi.org/10.5194/egusphere-egu24-451, 2024.

A.53
|
EGU24-21725
|
HS3.5
|
ECS
Valdrich Fernandes, Perry de Louw, Coen Ritsema, and Ruud Bartholomeus

Groundwater models are valuable tools for optimising decisions that influence groundwater flow. Spatially distributed models represent groundwater levels across the entire area from where essential information can be extracted, directly aiding in the decision-making process. In our previous study, we explored different machine learning (ML) models as faster alternatives to predict the increase in stationary groundwater head due to artificial recharge in unconfined aquifers while considering a wider spatial extent (832 columns x 1472 rows, totalling 765 km2) than previous ML groundwater models. The trained ML model accurately estimates the increase in groundwater head within 0.24 seconds, achieving a Nash-Sutcliffe efficiency of 0.95. This allows quick analysis of site suitability at potential recharge rates. This study extends the approach to incorporate seasonal variation in water availability, illustrating the concept of storing excess water during winter to meet heightened demands during summer, when water availability is minimal. Additionally, we quantify the impacts of the local properties, geohydrological and surface water network properties, on the storage capacity by training ML models on estimating the summer decay rate of stored water in hypothetical aquifer recharge sites.  

Among 720 hypothetical recharge sites, we vary its location, recharge rate and size to capture various combinations of local properties in the catchment. Artificial recharge is modeled using a MODFLOW-based groundwater model, representing the geo-hydrological properties and the surface water network in the Baakse Beek catchment in the Netherlands. The recharge is simulated from October 2011 till February 2012 with the remainder of the year simulated without any artificial recharge. Based on the modeled heads, the decay rate of stored water is calculated for the period until October. This calculated decay rate, in combination with the local properties are used to train and evaluate the ML model. The relative contributions of properties to the decay rate are quantified using the latest developments in explainable AI techniques. Techniques such as permutation importance and Ceteris paribus profiles not only help categorize the suitability of potential recharge sites but also quantify the relative contribution of each property. By leveraging these insights, water managers can make informed decisions regarding site improvement measures. 

How to cite: Fernandes, V., de Louw, P., Ritsema, C., and Bartholomeus, R.: Machine Learning Insights into Aquifer Recharge: Site suitability analysis in season water availability scenarios, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-21725, https://doi.org/10.5194/egusphere-egu24-21725, 2024.

A.54
|
EGU24-20579
|
HS3.5
|
ECS
|
Yucong Hu and Yan Jiang

Machine learning has long been restricted by the mystery of its black box, especially in the fields like geosciences that emphasizes clear expressions of mechanisms. To deal with that issue, we provided a fundamental framework combining two branches, clusters and regressions in machine learning, specifically, spectral clustering in unsupervised clustering methods and artificial neural networks in regression models, to resemble calculations in process-based models. With a case study of evapotranspiration, it was demonstrated that our framework was not only able to discern two processes, aerodynamics and energy, similar to the process-based model, i.e., Penman-Monteith formula, but also provided a third space for potential underrepresented process from canopy or ecosystems. Meanwhile, with only a few hundred of training data in most sites, the simulation of evapotranspiration achieved a higher accuracy (R2 of 0.92 and 0.82; RMSE of 12.41W/m2 and 8.11 W/m2 in training set and test set respectively) than commonly used machine learning approaches, like artificial neural networks in a scale of 100,000 training set (R2 of 0.85 and 0.81; RMSE of 42.33W/m2 and 46.73 W/m2). In summary, our method provides a new direction of hybridizing machine learning approaches and mechanisms for future work, which is able to tell mechanisms from a little amount of data, and thus could be utilized in validating the known and even exploring the unknown knowledge by providing reference before experiments and mathematical derivations.

How to cite: Hu, Y. and Jiang, Y.: Interpretably reconstruct physical processes with combined machine learning approaches, a case study of evapotranspiration, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-20579, https://doi.org/10.5194/egusphere-egu24-20579, 2024.