Producing accurate hourly streamflow forecasts in large basins is difficult without a distributed model to represent both streamflow routing through the river network and the spatial heterogeneity of land and weather conditions. HydroForecast is a theory-guided deep learning flow forecasting product that consists of short-term (hourly predictions out to 10 days), seasonal (10 day predictions out to a year), and daily reanalysis models. This work focuses primarily on the short-term model which has award winning accuracy across a wide range of basins.

In this work, we discuss the implementation of a novel distributed flow forecasting capability of HydroForecast, which splits basins into smaller sub-basins and routes flows from each subbasin to the downstream forecast points of interest. The entire model is implemented as a deep neural network allowing end-to-end training of both sub-basin runoff prediction and flow routing. The model's routing component predicts a unit hydrograph of flow travel time at each river reach and timestep allowing us to inspect and interpret the learned river routing and to seamlessly incorporate any upstream gauge data. 

We compare the accuracy of this distributed model to our original flow forecasting model at selected sites and discuss future improvements that will be made to this model.

How to cite: Lambl, D., Elkurdy, M., Butcher, P., Read, L. K., and Sampson, A. K.: Improving Data-Driven Flow Forecasting in Large Basins using Machine Learning to Route Flows, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4179, https://doi.org/10.5194/egusphere-egu23-4179, 2023.

Machine Learning and Deep Learning have been proving their potential for streamflow modelling in various studies. In particular, long short-term memory (LSTM) models showed exceptionally good results. However, machine learning models often are considered “black boxes” with limited interpretability. Explainable artificial intelligence (XAI) comprise methods that analyze the internal processes of the machine learning network and allow to have a glance in the “black box”. Most proposed XAI techniques are designed for the analysis of images, and there is currently only limited work on time series data available.

In our study, we applied various XAI algorithms including gradient-based methods (Saliency, InputXGradient, Integrated Gradient, GradientSHAP) but also perturbation-based methods (Feature Ablation, Feature Permutation) to compare their applicability for reasonable interpretation in the hydrological context. To our knowledge, only Integrated Gradient has been applied to a LSTM in hydrology so far. Gradient-based methods analyze the gradient of the output with respect to the input feature. Whereas perturbation-based methods gain information by altering or masking specific input features. The different methods were applied to a LSTM trained for the low-land Ems catchment in Germany, which has a major baseflow share of total streamflow.

We analyzed the results regarding their “timestep of influence”, which describes the amount of past days having importance for the prediction of streamflow at a particular day. All of the algorithms applied result in a comparable annual pattern, characterized by relatively small timesteps of influence in spring (wet season) and increasing timesteps of influence in summer and autumn (dry season). However, the range of the absolute days of attribution varies between the methods. In conclusion, all methods produces reasonable results and appear to be suitable for interpretation purposes.

Furthermore, we compare the results to ERA-5 reanalysis data and gained evidence that the LSTM recognizes soil water storage as the main driver for streamflow generation in the catchment: we found an inverse seasonality of soil moisture and timestep of influence.

How to cite: Ley, A., Bormann, H., and Casper, M.: Exploring different explainable artificial intelligence algorithms applied to a LSTM for streamflow modelling, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-3125, https://doi.org/10.5194/egusphere-egu23-3125, 2023.

Differentiable modeling has been introduced recently as a method to learn relationships from a combination of data and structural priors. This method uses end-to-end gradient tracking inside a process-based model to tune internal states and parameters along with neural networks, allowing us to learn underlying processes and spatial patterns. Hydrologic routing modules are typically needed to simulate flows in stem rivers downstream of large, heterogeneous basins, but obtaining suitable parameterization for them has previously been difficult. In this work, we apply differentiable modeling in the scope of streamflow prediction by coupling a physically-based routing model (which computes flow velocity and discharge in the river network given upstream inflow conditions) to neural networks which provide parameterizations for Manning’s river roughness parameter (n). This method consists of an embedded Neural Network (NN), which uses (imperfect) DL-simulated runoffs and reach-scale attributes as forcings and inputs, respectively, entered into the Muskingum-Cunge method and trained solely on downstream discharge. Our initial results show that while we cannot identify channel geometries, we can learn a parameterization scheme for roughness that follows observed n trends. Training on a short sample of observed data showed that we could obtain highly accurate routing results for the training and inner, untrained gages. This general framework can be applied to small and large scales to learn channel roughness and predict streamflow with heightened interpretability. 

How to cite: Bindas, T., Tsai, W.-P., Liu, J., Rahmani, F., Feng, D., Bian, Y., Lawson, K., and Shen, C.: Improving large-basin streamflow simulation using a modular, differentiable, learnable graph model for routing, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16658, https://doi.org/10.5194/egusphere-egu23-16658, 2023.

Streamflow predictions are a vital tool for detecting flood and drought events. Such predictions are even more critical to Sub-Saraharan African regions that are vulnerable to the increasing frequency and intensity of such events. These regions are sparsely gauged, with few available gauging stations that are often plagued with missing data due to various causes, such as harsh environmental conditions and constrained operational resources. 

This work presents a novel workflow for predicting streamflow in the presence of missing gauge observations. We leverage bias correction of the GEOGloWS ECMWF streamflow service (GESS) forecasts for missing data imputation and predict future streamflow using the state-of-the-art Temporal Fusion transformers at ten river gauging stations in the Benin Republic.

We show by simulating missingness in a testing period that GESS forecasts have a significant bias that results in poor imputation performance over the ten Beninese stations. Our findings suggest that overall bias correction by Elastic Net and Gaussian Process regression achieves superior performance relative to traditional imputation by established methods such as Random Forest, k-Nearest Neighbour, and GESS lookup. We also show that the Temporal Fusion Transformer yields high predictive skill and further provides explanations for predictions through the weights of its attention mechanism. The findings of this work provide a basis for integrating Global streamflow prediction model data and state-of-the-art machine learning models into operational early-warning decision-making systems (e.g., flood/ drought alerts) in resource-constrained countries vulnerable to drought and flooding due to extreme weather events.

How to cite: Mbuvha, R., Adounkpe, P. J. Y., Houngnibo, M. C. M., and Newlands, N.: A Novel Workflow for Streamflow Prediction in the Presence of Missing Gauge Observations, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5736, https://doi.org/10.5194/egusphere-egu23-5736, 2023.

Neural Ordinary Differential Equation (ODE) models have demonstrated high potential in providing accurate hydrologic predictions and process understanding for single catchments (Höge et al., 2022). Neural ODEs fuse a neural network model core with a mechanistic equation framework. This hybrid structure offers both traceability of model states and processes, like in conceptual hydrologic models, and the high flexibility of machine learning to learn and refine model interrelations. Aside of the functional dependence of internal processes on driving forces, like of evapotranspiration on temperature, Neural ODEs are also able to learn the effect of catchment-specific attributes, e.g. land cover types, on processes when being trained over multiple basins simultaneously.

We demonstrate the performance of a generic Neural ODE architecture in a hydrologic large-sample setup with respect to both predictive accuracy and process interpretability. Using several hundred catchments, we show the capability of Neural ODEs to learn the general interplay of catchment-specific attributes and hydrologic drivers in order to predict discharge in out-of-sample basins. Further, we show how functional relations learned (encoded) by the neural network can be translated (decoded) into an interpretable form, and how this can be used to foster understanding of processes and the hydrologic system.

Höge, M., Scheidegger, A., Baity-Jesi, M., Albert, C., & Fenicia, F.: Improving hydrologic models for predictions and process understanding using Neural ODEs. Hydrol. Earth Syst. Sci., 26, 5085-5102, https://hess.copernicus.org/articles/26/5085/2022/

How to cite: Höge, M., Scheidegger, A., Baity-Jesi, M., Albert, C., and Fenicia, F.: Neural ODE Models in Large-Sample Hydrology, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-6466, https://doi.org/10.5194/egusphere-egu23-6466, 2023.

Although deep learning (DL) models have shown extraordinary performance in hydrologic modeling, they are still hard to interpret and not able to predict untrained hydrologic variables due to lacking physical meanings and constraints. This study established hybrid differentiable models (namely the delta models) with regionalized parameterization and learnable structures based on a DL-based differentiable parameter learning (dPL) framework. The simulation experiments on both US and global basins demonstrate that the delta models can approach the performance of the state-of-the-art long short-term memory (LSTM) network on discharge prediction. Different from the pure data-driven LSTM model, the delta models can output a full set of hydrologic variables not used as training targets. The evaluation with independent data sources showed that the delta models, only trained on discharge observations, can also give decent predictions for ET and baseflow. The spatial extrapolation experiments showed that the delta models can surpass the performance of the LSTM model for predictions in large ungauged regions in terms of the daily hydrographic metrics and multi-year trend prediction. The spatial patterns of the parameters learned by the delta models remain remarkably stable from the in-sample to spatial out-of-sample predictions, which explains the robustness of the delta models for spatial extrapolation. More importantly, the proposed modeling framework enables directly learning new relations between intermediate variables from large observations. This study shows that the model performance and physical meanings can be balanced with the differentiable modeling approach which is promising to large-scale hydrologic prediction and knowledge discovery.

How to cite: Feng, D. and Shen, C.: A differentiable modeling approach to systematically integrating deep learning and physical models for large-scale hydrologic prediction and knowledge discovery, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16947, https://doi.org/10.5194/egusphere-egu23-16947, 2023.

Simulation of the catchment rainfall-runoff transformation with physically based watershed models is a traditional way to predict streamflow and other hydrological variables at catchment scales. However, the calibration of such models requires large data inputs and computational power and contains many parameters which are often impossible to constrain or validate. An alternative approach is to use data-driven machine learning for streamflow prediction.

In the past few years, LSTM (long short-term memory) models and its variants have been explored in rainfall-runoff modelling. Typical applications use daily climate variables as inputs and model the rainfall-runoff transformation processes with different timescales of memory. This is especially useful as delays in runoff production by snow accumulation and melt, soil water storage, evapotranspiration, etc., can be included. In contrast to feed-forward ANNs (artificial neural networks), LSTMs are capable of maintaining the sequential temporal order of inputs, and compared to RNNs (recurrent neural networks), of learning the long-term dependencies. [1]

However, current work on LSTMs mostly focuses on the USA, the UK and Brazil, where CAMELS datasets are available [1, 2, 3]. Catchments at higher altitudes with snow-driven dynamics and sometimes glaciers are present in small number in these datasets (if at all). Systematic applications of LSTMs for streamflow prediction in climates where a significant part of the catchments are snow and ice dominated are missing. In this work, an FS-LSTM (fast slow-LSTM) previously applied in Brazil is adapted for Swiss catchments to fill this gap [3]. The FS-LSTM explored builds on the work of Hoedt et al. (2021) that imposed mass constraints on an LSTM, called MC-LSTM [4]. FS-LSTM adds a fast and slow part for streamflow, containing rainfall and soil moisture respectively. We will discuss benchmark results against an existing semi-distributed conceptual model widely used in Switzerland for streamflow simulation [5].

References:

[1]: Kratzert et al., Rainfall-runoff modelling using Long Short-Term Memory (LSTM) networks, 2018.

[2]: Lees et al., Hydrological concept formation inside long short-term memory (LSTM) networks, 2022.

[3]: Quinones et al., Fast-Slow Streamflow Model Using Mass-Conserving LSTM, 2021.

[4]: Hoedt et al., MC-LSTM: Mass-Conserving LSTM, 2021.

[5]: Viviroli et al., An introduction to the hydrological modelling system PREVAH and its pre- and post-processing-tools, 2009.

How to cite: Lott, C., Martins, L., Weiss, J., Brunschwiler, T., and Molnar, P.: LSTMs for Hydrological Modelling in Swiss Catchments, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14399, https://doi.org/10.5194/egusphere-egu23-14399, 2023.

The objective function plays an important role in the training process for deep learning models, since it largely determines the trained values of the model parameters and influences the model performance. In this study, we establish two application-orientated objective functions, namely high flow balance error (HFBE) and transformed mean absolute percentage error (MAPE*), for the forecasts of high flows and low flows, respectively, in the LSTM model. We examine the strength and weakness of these streamflow forecast models trained on HFBE, MAPE* and mean square error (MSE) based on multiple performance metrics. Furthermore, we propose the objective function-based ensemble model (OEM) framework that integrates the models trained on different objective functions, so as to take advantages of the trained models focusing on different aspects of streamflow and thus achieve a better overall performance. Our results in 273 catchments over USA show that the models trained on HFBE can alleviate underestimation in high flows existing in the models trained on MSE, and perform remarkably better for high flows. It is also found that the models trained on MAPE* outperform the other two models in low flow forecast, no matter what algorithm is used for the model establishment. By incorporating the three models trained on HFBE, MAPE* and MSE, respectively, our proposed OEM performs well in the forecasts of both high flows and low flows, and realistically capture the mean and the variability of the observational streamflow under different scenarios under a variety of hydrometeorological conditions. This study highlights the necessity of applying application-orientated objective functions for given projects and the great potential of the ensemble learning methods for multi-optimization in hydrological modeling.

How to cite: Wang, D.: The role of ensemble learning in multi-optimization for streamflow prediction, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-5044, https://doi.org/10.5194/egusphere-egu23-5044, 2023.

Deep learning has become the de facto standard for streamflow simulation. While there are examples of deep learning based streamflow forecast models (e.g., 1-5), the majority of the development and research has been done with hindcast models. The primary challenge in using deep learning models for forecasting (e.g., flood forecasting) is that the meteorological input data are drawn from different distributions in hindcast vs. forecast. The (relatively small) amount of research that has been done on deep learning streamflow forecasting has largely used an encoder-decoder approach to account for forecast distribution shifts. This is, for example, what Google’s operational flood forecasting model uses [4]. 

In this work we show that the encoder-decoder approach results in artifacts in forecast trajectories that are not detectable with standard hydrological metrics, but which can cause forecasts to have incorrect trends (e.g., rising when they should be falling and vice-versa).  We solve this problem using regularized embeddings, which remove forecast artifacts without harming overall accuracy. 

Perhaps more importantly, input embeddings allow for training models on spatially and/or temporally incomplete meteorological inputs, meaning that a single model can be trained using input data that does not exist everywhere or does not exist during the entire training or forecast period. This allows models to learn from a significantly larger training data set, which is important for high-accuracy predictions. It also allows large (e.g., global) models to learn from local weather data. We demonstrate how and why this is critical for state-of-the-art global-scale streamflow forecasting. 

• Franken, Tim, et al. An operational framework for data driven low flow forecasts in Flanders. No. EGU22-6191. Copernicus Meetings, 2022.
• Kao, I-Feng, et al. "Exploring a Long Short-Term Memory based Encoder-Decoder framework for multi-step-ahead flood forecasting." Journal of Hydrology 583 (2020): 124631.
• Liu, Darong, et al. "Streamflow prediction using deep learning neural network: case study of Yangtze River." IEEE access 8 (2020): 90069-90086.
• Nevo, Sella, et al. "Flood forecasting with machine learning models in an operational framework." Hydrology and Earth System Sciences 26.15 (2022): 4013-4032.
• Girihagama, Lakshika, et al. "Streamflow modelling and forecasting for Canadian watersheds using LSTM networks with attention mechanism." Neural Computing and Applications 34.22 (2022): 19995-20015.

How to cite: Nearing, G., Gauch, M., Klotz, D., Kratzert, F., Metzger, A., Shalev, G., Shenzis, S., Tekalign, T., Weitzner, D., and Gilon, O.: From Hindcast to Forecast with Deep Learning Streamflow Models, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-16974, https://doi.org/10.5194/egusphere-egu23-16974, 2023.

Neural networks belong to the best available methods for numerous hydrological model challenges. However, although they have shown to outperform classical hydrological models in several applications there is still some doubt whether neural networks are, despite their excellent interpolation skills, capable to make predictions beyond the support of the training data. This study addresses this issue and proposes an approach to infer the ability of neural network to predict unusual, extreme system states. We show how we can use the concept of data surprise and model surprise in a complementary manner to assess which unusual events a neural network can predict, which it can predict but only with additionally data and which it cannot predict at all hinting toward the wrong model choice or towards an incomplete description of the data.

How to cite: Loritz, R. and Gupta, H.: Extrapo… what? Predictions beyond the support of the training data, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1526, https://doi.org/10.5194/egusphere-egu23-1526, 2023.

Improving the understanding of processes is vital to hydrological modeling. One key challenge is how to extract interpretable information that can describe the complex hydrological system from the growing number of observation data to advance our understanding of processes and modeling. To address the problem, we propose a data-driven framework to discover coordinate transformation, which transfers original observations to a reduced-dimension system. The framework combines deep learning method with sparse regression to approximate the specific hydrological process: deep learning methods have a rich representation to promote generalization, and sparse regression can sparsely identify parsimonious models to promote interpretability. By doing so, we can identify the essential latent variables under a physically meaning-wise coordinate system where the hydrological processes are linearly and sparsity represented to capture the behavior of the system from observations. To demonstrate the framework, we focus on the evaporation process. The relationships between potential evaporation and climate variables including long/short wave radiation, air temperature, air pressure, relative humidity, and wind speed are quantified. The connection between the climate variables and coordinates components extracted are evaluated to capture the pattern of climate variables in the component space. The robustness and statistical stability of the framework is examined based on distributed observations from FluxNet towers over North America. The resulting modeling framework shows the potential of deep learning methods for improving our knowledge of the hydrological system.

How to cite: Hu, X., Tuo, Y., and Disse, M.: Deep learning based coordinates transformations for improving process understanding in hydrological modeling system, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-14631, https://doi.org/10.5194/egusphere-egu23-14631, 2023.

The high computational cost of detailed numerical models for flood simulation hinders their use in real-time and limits uncertainty quantification. Deep-learning surrogates have thus emerged as an alternative to speed up simulations. However, most surrogate models currently work only for a single topography, meaning that they need to be retrained for different case studies, ultimately defeating their purpose. In this work, we propose a graph neural network (GNN) inspired by the shallow water equations used in flood modeling, that can generalize the spatio-temporal prediction of floods over unseen topographies. The proposed model works similarly to finite volume methods by propagating the flooding in space and time, given initial and boundary conditions. Following the Courant-Friedrichs-Lewy condition, we link the time step between consecutive predictions to the number of GNN layers employed in the model. We analyze the model's performance on a dataset of numerical simulations of river dike breach floods, with varying topographies and breach locations. The results suggest that the GNN-based surrogate can produce high-fidelity spatio-temporal predictions, for unseen topographies, unseen breach locations, and larger domain areas with respect to the training ones, while reducing computational times.

How to cite: Bentivoglio, R., Isufi, E., Jonkman, S. N., and Taormina, R.: On the generalization of hydraulic-inspired graph neural networks for spatio-temporal flood simulations, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-12952, https://doi.org/10.5194/egusphere-egu23-12952, 2023.

Accurate flood frequency analysis is essential for developing effective flood management strategies and designing flood protection infrastructure, but it is challenging due to the complex, nonlinear hydrological system. In regional flood frequency analysis (RFFA), the flood quantiles at ungauged sites can be estimated by establishing a relationship between interdependent physio-meteorological variables and observed flood quantiles at gauge sites in the region. However, this regional approach implies a loss of information due to the prior aggregation of hydrological data at gauged locations and can be difficult for data-sparse regions due to limited data. In this study, we evaluated an alternate approach or path for RFFA in two case studies: a data-sparse region in India and a data-dense region in the USA. In this approach, daily streamflow is predicted first using a deep learning-based hydrological model, and then flood quantiles are estimated from the predicted daily streamflow using statistical methods. We compared the results obtained using this alternate approach to those from the traditional RFFA technique, which used the Random Forest (RF) and eXtreme Gradient Boosting (XGB) algorithms to model the nonlinear relationship between flood quantiles and relevant physio-meteorological predictor variables such as meteorological forcings, topography, land use, and soil properties. The results showed that the alternate approach produces more reliable results with the least mean absolute error and higher coefficient of determination in the data-sparse region. In the data-dense region, both traditional and alternate approaches produced comparable results. However, the alternate approach has the advantage of being flexible and providing the complete time series of daily flow at the ungauged location, which can be used to estimate other flow characteristics, develop flow duration curves, or estimate flood quantiles of any return period without creating a separate traditional RFFA model. This study shows that the alternate approach can provide accurate flood frequency estimates in data-sparse regions, offering a promising solution for flood management in these areas.

How to cite: Mangukiya, N. K. and Sharma, A.: Evaluating Machine Learning Approach for Regional Flood Frequency Analysis in Data-sparse Regions, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-1278, https://doi.org/10.5194/egusphere-egu23-1278, 2023.

How to cite: Thibaut, R., Ferré, T., Laloy, E., and Hermans, T.: Sequential optimization of temperature measurements to estimate groundwater-surface water interactions, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4137, https://doi.org/10.5194/egusphere-egu23-4137, 2023.

How to cite: de FLEURY, M., Kergoat, L., Brandt, M., Fensholt, R., Kariryaa, A., Kovács, G. M., Horion, S., and Grippa, M.: Deep learning for mapping water bodies in the Sahel, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-7347, https://doi.org/10.5194/egusphere-egu23-7347, 2023.

Terrestrial water storage (TWS) anomalies from Gravity Recovery and Climate Experiment (GRACE) and its follow on GRACE-FO satellite missions provide a unique opportunity to measure the impact of different climate extremes and human intervention on water use at regional and continental scales. However, temporal gaps within GRACE and GRACE-FO mission (GRACE: 20 months, between GRACE and GRACE-FO: 11 months and GRACE-FO: 2 months) pose difficulties in analyzing spatiotemporal variations in TWS. In this study, Convolutional Long Short-Term Memory Neural Networks (CNN-LSTM) model was developed to fill these gaps and reconstruct the TWS for the Indian subcontinent (April 2002-July 2022). Various meteorological and climatic variables, such as precipitation, temperature, run-off, evapotranspiration, and vegetation, have been integrated to predict GRACE TWS. The performance of the models was evaluated with the help of Pearson’s correlation coefficient (PR), Nash-Sutcliffe efficiency (NSE), and Normalised Root Mean Square Error (NRMSE). Results indicate that the CNN-LSTM model yielded a mean PR of 0.94 and 0.89, NSE of 0.87 and 0.8, and NRMSE of 0.075 and 0.101 on training and testing, respectively. Overall, the CNN-LSTM achieved good performance except in the northwestern region of India, which showed a relatively poor performance might be due to high anthropogenic activity and arid climatic conditions. Further reconstructed time series were used to study the Spatiotemporal variations of TWS over the Indian Subcontinent.

Keywords: GRACE; Deep Learning; TWSA; Indian subcontinent

How to cite: Moudgil, P. S. and Rao, G. S.: Filling Temporal Gaps within and between GRACE and GRACE-FO Terrestrial Water Storage Changes over Indian Sub-Continent using Deep Learning., EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-8218, https://doi.org/10.5194/egusphere-egu23-8218, 2023.

Reliable and accurate precipitation estimations are a crucial hydrological parameter for various applications, including managing water resources, drought monitoring and natural hazard prediction. The two main approaches for estimating precipitation from satellite data are the top-down and bottom-up. The top-down approach uses data from Geostationary and Low Earth Orbiting satellites to infer precipitation from atmosphere and cloud information, while the bottom-up approach estimates precipitation using soil moisture observations, e.g.  the SM2RAIN algorithm. The main difference between these approaches is that the top-down approach is a more direct method of measuring precipitation that estimates it instantaneously, which may lead to underestimation, while the bottom-up approach measures accumulated rainfall with more reliable precipitation estimation between two consecutive SM measurements. In this study, we develop the deep convolutional neural networks (CNN) algorithm to combine the top-down and bottom-up approaches for estimating precipitation using the satellite level 1 products including the satellite backscatter information from the Advanced SCATterometer (ASCAT), infrared (IR) and water vapor (WV) channels from geostationary satellites. This algorithm is assessed at 0.1° spatial and daily temporal resolution over Italy for the period of 2019-2021. The results show that the developed model improves the accuracy of precipitation estimation. Additionally, it indicates that there is a significant potential for global precipitation estimation using this model.

How to cite: Mosaffa, H., Filippucci, P., Ciabatta, L., Massari, C., and Brocca, L.: Application of deep convolutional neural networks for precipitation estimation through both top-down and bottom-up approaches, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-15575, https://doi.org/10.5194/egusphere-egu23-15575, 2023.

IMERG is a global satellite-based precipitation dataset, produced by NASA. It has provided valuable rainfall information to facilitate the design or the operation of the disaster and risk management worldwide. In operation, NASA offers three types of IMERG Level 3 (L3) products, with different levels of trade-offs in terms of time latency and accuracy. These are Early run (4-hour latency), Late run (14-hour latency) and Final run(3.5-month latency). The final-run product integrates multi-sensor retrievals and provides the highest-quality precipitation estimates among three IMERG products. It however suffers from a long processing latency, which hinders its applicability to near real-time applications. In the past 10 years, deep learning techniques have made significant breakthroughs in various scientific fields, including short-term rainfall forecasting. Deep learning models have shown to have the potential to learn the complex variations in weather systems and to outperform the Numerical Weather Prediction (NWP) in terms of short lead-time predictability and the required computational resources for operation.

In this research, we would like to explore the potential of deep learning (DL) in generating high-quality satellite-based precipitation product with low latency. More specifically, we investigate if DL models can learn the difference between Final- and Early-run products, and thus predict a Final-run-like product using Early-run product as input. Low-latency yet high-quality IMERG precipitation product can be therefore obtained. Various DL techniques are being tested in this work, including Auto-Encoder(AE), ConvLSTM and Deep Generative model. IMERG data between 2018 and 2020 over a rectangular area centred in the UK is used for model training and testing, and ground rain gauge records will be used to evaluate the performance of the original and predicted products. This pilot includes both ocean and land regions, which enables the comparison of the model performance between two different surface conditions. Preliminary analysis suggests that given patterns do exist in the differences between Early- and Final-run products, and the capacity of the selected DL models to learn the differences will be further investigated. The proposed work is of great potential to improve the applicability of IMERG products in an operational context.

How to cite: Hung, H. T. and Wang, L.-P.: IMERG Run Deep: Can we produce a low-latency IMERG Final run product with a deep learning based prediction model?, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-4887, https://doi.org/10.5194/egusphere-egu23-4887, 2023.