HS3.1

With the increasing water requirements and weather extremes, effective planning and management for water issues has been of great concern over the past decades. Accurate and reliable streamflow forecasting is a critical step for water resources supply and prevention of natural disasters. In this study, we developed a hybrid model (namely GMM-XGBoost), coupling extreme gradient boosting (XGBoost) with Gaussian mixture model (GMM), for monthly streamflow forecasting. The proposed model is based on the principle of modular model, where a complex problem is divided into several simple ones. GMM was applied to cluster streamflow into several groups, using the features selected by a tree-based method. Then, each group was used to fit several single XGBoosts. And the prediction is a weighted average of the single models. Two streamflow datasets were used to evaluate the performance of the proposed model. The prediction accuracy of GMM-XGBoost was compared with that of support vector machine (SVM) and standalone XGBoost. The results indicated that although all three models yielded quite good performance on one-month ahead forecasting with high Nash-Sutclitte efficiency coefficient (NSE) and low root mean squared error (RMSE), GMM-XGBoost provided the best accuracy with significant improvement of forecasting accuracy. It can be inferred from the results that (1) XGBoost is applicable for streamflow forecasting, and in general, performs better than SVM; (2) the cluster analysis-based modular model is helpful in improving accuracy; (3) the proposed GMM-XGBoost model is a superior alternative, which can provide accurate and reliable predictions for optimal water resources management.

Note: This study has been published in Journal of Hydrology (Ni, L., Wang, D., Wu, J., Wang, Y., Tao, Y., Zhang, J. and Liu, J., 2020. Streamflow forecasting using extreme gradient boosting model coupled with Gaussian mixture model. Journal of Hydrology, 586.).

How to cite: Ni, L., Wang, D., Wu, J., and Wang, Y.: A hybrid model coupling extreme gradient boosting model with Gaussian mixture model for streamflow forecasting, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-3590, https://doi.org/10.5194/egusphere-egu21-3590, 2021.

Long short-term memory (LSTM) networks are able to learn and replicate the relationships of multiple climate and hydrological temporal variables, and therefore are theoretically suitable for data driven modelling and forecasting of rainfall-runoff behavior. However, they inherit some prediction errors occasionally found in data-driven models: phase shift errors, oscillations and total failures. The phase shift error is a particularly significant challenge due to its occurrence when using hourly precipitation and runoff data for catchments with short response times.

In order to detect and eliminate these errors, we investigated four approaches, of which the first two are of structural nature, while the last two modify the input time series by certain transformations:
1. The use of encoder-decoder architectures for LSTM networks.
2. Offsetting the start of the flood forecast to the forecast time step of interest.
3. The inversion of the input time series.
4. Including subsequently observed precipitation data as a “best precipitation forecast”.

We tested the four approaches on five different pilot catchments located in Saxony, Germany with relatively short response times. The results show no advantage of the structural approaches. In contrast, the modification of the input time series shows potential for improving the predictive quality of flood forecasting in a potential operational application.

How to cite: Morgenstern, T., Pahner, S., Mietrach, R., and Schütze, N.: Flood forecasting in small catchments using deep learning LSTM networks, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-15072, https://doi.org/10.5194/egusphere-egu21-15072, 2021.

In this contribution, I will suggest an approach to build models as ordered and connected collections of multivariate, discrete probability distributions (dpd's). This approach can be seen as a Machine-Learning (ML) approach as it allows very flexible learning from data (almost) without prior constraints. Models can be built on dpd's only (fully data-based model), but they can also be included into existing process-based models at places where relations among data are not well-known (hybrid model). This provides flexibility for learning similar to including other ML approaches - e.g. Neural Networks - into process-based models, with the advantage that the dpd's can be investigated and interpreted by the modeler as long as their dimensionality remains low. Models based on dpd's are fundamentally probabilistic, and model responses for out-of-sample situations can be assured by dynamically coarse-graining the dpd's: The farther a predictive situation is from the learning situations, the coarser/more uncertain the prediction will be, and vice versa.

I will present the main elements and steps of such dpd-based modeling at the example of several systems, ranging from simple deterministic (ideal spring) to complex (hydrological system), and will discuss the influence of i) the size of the available training data set, ii) choice of the dpd priors, and iii) binning choices on the models' predictive power.

How to cite: Ehret, U.: Hybrid modeling using multivariate, discrete probability distributions, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-2866, https://doi.org/10.5194/egusphere-egu21-2866, 2021.

Prediction of vegetation transpiration (T) is of increasing importance in water resources management and agricultural practices, in particular to facilitate precision irrigation. Traditional evapotranspiration (ET) partitioning dual source modeling requires an extensive array of ground-level parameters and needs model correction and calibration to attain model certainty. In response, a quick and low-cost method is described to predict T using artificial intelligence (AI) modeling based on meteorological factors, status of crop growth factors and soil parameters. This study compares Random Forest (RF) and Support Vector Regression (SVR) in building AI models using three years (2014–2017) of continuous high-resolution monitoring data in a cabbage farmland.Input data included air temperature (Ta), solar radiation (Ra), relative humidity (RH), vapor pressure deficit(VPD), wind speed (Ws), soil moisture (SM), vegetation height (H), and leaf area index (LAI). The results show that soil surface resistance calculations by Monte Carlo iterative method and vegetation stomatal resistance calculations and carbon dioxide concentration and emission, improve performance of the original Shuttleworth–Wallace(S-W) model. In addition, the AI model indicates Ta and Ra are essential inputs for both model types. When there are sufficient observation data, or only lacking soil and vegetation data, the RF model is recommended for use. When there are only limited data or lack of critical Ta and Ra data, the SVR model is the preferred model. Scientific guidance is provided for agriculture precision irrigation, indicating which AI model can best estimate T and water demand for irrigation planning and water management.

How to cite: Li, H., Chen, H., Huang, J. J., McBean, E., Zhang, J., Gao, J., and Lan, Z.: Partitioning of daily evapotranspiration using a modified shuttleworthwallace model, random Forest and support vector regression, for a cabbage farmland, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4024, https://doi.org/10.5194/egusphere-egu21-4024, 2021.

In recent years, the application of model ensembles has received increasing attention in the hydrological modelling community due to the interesting results reported in several studies carried out in different parts of the world. The main idea of these approaches is to combine the results of the same hydrological model or a number of different hydrological models in order to obtain more robust, better-fitting models, reducing at the same time the uncertainty in the predictions. The techniques for combining models range from simple approaches such as averaging different simulations, to more complex techniques such as least squares, genetic algorithms and more recently artificial intelligence techniques such as Artificial Neural Networks (ANN).

Despite the good results that model ensembles are able to provide, the models selected to build the ensemble have a direct influence on the results. Contrary to intuition, it has been reported that the best fitting single models do not necessarily produce the best ensemble. Instead, better results can be obtained with ensembles that incorporate models with moderate goodness of fit. This implies that the selection of the single models might have a random component in order to maximize the results that ensemble approaches can provide.

The present study is carried out using hydrological data on an hourly scale between 2008 and 2015 corresponding to the Mandeo basin, located in the Northwest of Spain. In order to obtain 1000 single models, a hydrological model was run using 1000 sets of parameters sampled randomly in their feasible space. Then, we have classified the models in 3 groups with the following characteristics: 1) The 25 single models with highest Nash-Sutcliffe coefficient, 2) The 25 single models with the highest Pearson coefficient, and 3) The complete group of 1000 single models.

The ensemble models are built with 5 models as the input of an ANN and the observed series as the output. Then, we applied the Random-Restart Hill-Climbing (RRHC) algorithm choosing 5 random models in each iteration to re-train the ANN in order to identify a better ensemble. The algorithm is applied to build 50 ensembles in each group of models. Finally, the results are compared to those obtained by optimizing the model using a gradient-based method by means of the following goodness-of-fit measures: Nash-Sutcliffe (NSE) coefficient, adapted for high flows Nash-Sutcliffe (HF−NSE), adapted for low flows Nash-Sutcliffe (LF−W NSE) and coefficient of determination (R2).

The results show that the RRHC algorithm can identify adequate ensembles. The ensembles built using the group of models selected based on the NSE outperformed the model optimized by the gradient method in 64 % of the cases in at least 3 of 4 coefficients, both in the calibration and validation stages. Followed by the ensembles built with the group of models selected based on the Pearson coefficient with 56 %. In the case of the third group, no ensembles were identified that outperformed the gradient-based method. However, the most part of the ensembles outperformed the 1000 individual models.

Keywords: Multi-model ensemble; Single-model ensemble; Artificial Neural Networks; Hydrological Model; Random-restart Hill-climbing

How to cite: Farfán-Durán, J. F. and Cea, L.: Building hydrological single-model ensembles using artificial neural networks and a combinatorial optimization approach, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-8256, https://doi.org/10.5194/egusphere-egu21-8256, 2021.

Analog methods (AMs) are statistical downscaling methods often used for precipitation prediction in different contexts, such as operational forecasting, past climate reconstruction of climate change impact studies. It usually relies on predictors describing the atmospheric circulation and the moisture content of the atmosphere to sample similar meteorological situations in the past and establish a probabilistic forecast for a target date. AMs can be based on outputs from numerical weather prediction models in the context of operational forecasting or outputs from climate models in climatic applications.

AMs can be constituted of multiple predictors organized in different subsequent levels of analogy that refines the selection of similar situations. The development of such methods is usually a manual process where some predictors are assessed in different structures. As most AMs use multiple predictors, a comprehensive assessment of all combinations becomes quickly impossible. The selection of predictors in the application of the AM often builds on previous work and does not evolve much. However, the climate models providing the predictors evolve continuously and new variables might become relevant to be considered in AMs. Moreover, the best predictors might change from one region to another or for another predictand of interest. There is a need for a method to automatically explore potential variables for AMs and to extract the ones that are relevant for a predictand of interest.

We propose using genetic algorithms (GAs) to proceed to an automatic selection of the predictor variables along with all other parameters of the AM. We even let the GAs automatically pick the best analogy criteria, i.e. the metric that quantifies the analogy between two situations. The first test consisted of letting the GAs select the single best variable to predict daily precipitation for each of 25 selected catchments in Switzerland. The results showed great consistency in terms of spatial patterns and the underlying meteorological processes. Then, different structures were assessed by varying the number of levels of analogy and the number of variables per level. Finally, multiple optimizations were conducted on the 25 catchments to identify the 12 variables that provide the best prediction when considered together.

How to cite: Horton, P. and Martius, O.: Automatic input variable selection for analog methods using genetic algorithms, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-9375, https://doi.org/10.5194/egusphere-egu21-9375, 2021.

The socio-economic development of a country depends mainly on adequate integrated water resources management (IWRM). Sectors such as mining and agriculture are two main economic activities in Bolivia, that negatively impact the water resource quality and availability. Also, every year, floods and droughts hit the most vulnerable populations in different regions of Bolivia. Floods represent the greatest hydroclimatological risk factor in the country along with landslides caused by heavy precipitation. Along with these challenges in the country, there is also inefficient water treatment for water supply which can lead to other problems like diseases. Nowadays, the media such as newspapers, television, radio, report on these problems, in terms of water resources, which are experienced year after year in the country. Furthermore, due to advances in technology, this information can be found digitally. In the same way, people have made use of social networks, such as twitter, to express their opinion on a specific topic. The type of information found both in the media and in social networks is called qualitative information.

This digital information will be extracted using web crawling and web scrapping techniques that allow the process to be automated. This process is performed by applying keywords in the context of water resources in Bolivia, such as names of different water bodies in a basin. Once the information has been extracted, it will be transformed into a quantitative form, in such a way that it is useful for planning and decision-making processes of IWRM in Bolivia.

The purpose of this research is focused on the application of Natural Language Processing in the digital information found for three hydrological basins located in Bolivia, in order to recognize how Bolivian society relates the management of water resources. These hydrological basins are La Paz - Choqueyapu, Tupiza and Pampa – Huari. Initially, the digital information that will be studied in this research consists of three Bolivian newspapers and the information found on Twitter. The application of a sentiment analysis classification model in Python language programming is developed. In order to preserve the semantic information and the different words in the text, Word2Vec model will be used. The extracted digital information is pre-processed, eliminating empty words that do not add sentiments to a text and punctuation marks. Once the information is pre-processed, it is divided into two types, training and testing. The training data will be used to train the Word2Vec model. The result of the model consists of a value that determines the positive, neutral or negative sentiment of the text. Once the model is trained, the testing data that has not been used will be applied in order to evaluate the performance of the model.

This research helps to identify key elements, actors, frequent words related to IWRM, factors related to river health and improve the concept of citizen science. The results are mapped by geolocation, as a frequency distribution considering the digital perception (sentiment analysis) found and the frequency in which a topic is mentioned in the analysed digital information.

How to cite: Gonzalez Ayala, C. A., Duarte Prieto, S., Escalera, A., Corzo Perez, G., Angarita, H., and Santos Granados, G.: Natural Language Processing In Integrated Water Resources Management. Case Study: Three Bolivian River Basins, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-13198, https://doi.org/10.5194/egusphere-egu21-13198, 2021.

Bogotá’s River Basin, it’s an important basin in Cundinamarca, Colombia’s central region. Due to the complexity of the dynamical climatic system in tropical regions, can be difficult to predict and use the information of GCMs at the basin scale. This region is especially influenced by ENSO and non-linear climatic oscillation phenomena. Furthermore, considering that climatic processes are essentially non-linear and possibly chaotic, it may reduce the effectiveness of downscaling techniques in this region. 

In this study, we try to apply chaotic downscaling to see if we could identify synchronicity that will allow us to better predict. It was possible to identify clearly the best time aggregation that can capture at the best the maximum relations between the variables at different spatial scales. Aside this research proposes a new combination of multiple attractors. Few analyses have been made to evaluate the existence of synchronicity between two or more attractors. And less analysis has considered the chaotic behaviour in attractors derived from climatic time series at different spatial scales. 

Thus, we evaluate general synchronization between multiple attractors of various climate time series. The Mutual False Nearest Neighbours parameter (MFNN) is used to test the “Synchronicity Level” (existence of any type of synchronization) between two different attractors. Two climatic variables were selected for the analysis: Precipitation and Temperature. Likewise, two information sources are used: At the basin scale, local climatic-gauge stations with daily data and at global scale, the output of the MPI-ESM-MR model with a spatial resolution of 1.875°x1.875° for both climatic variables (1850-2005). In the downscaling process, two RCP (Representative Concentration Pathways)  scenarios are used, RCP 4.5 and RCP 8.5.

For the attractor’s reconstruction, the time-delay is obtained through the  Autocorrelation and the Mutual Information functions. The False Nearest Neighbors method (FNN) allowed finding the embedding dimension to unfold the attractor. This information was used to identify deterministic chaos at different times (e.g. 1, 2, 3 and 5 days) and spatial scales using the Lyapunov exponents. These results were used to test the synchronicity between the various chaotic attractor’s sets using the MFNN method and time-delay relations. An optimization function was used to find the attractor’s distance relation that increases the synchronicity between the attractors.  These results provided the potential of synchronicity in chaotic attractors to improve rainfall and temperature downscaling results at aggregated daily-time steps. Knowledge of loss information related to multiple reconstructed attractors can provide a better construction of downscaling models. This is new information for the downscaling process. Furthermore, synchronicity can improve the selection of neighbours for nearest-neighbours methods looking at the behaviour of synchronized attractors. This analysis can also allow the classification of unique patterns and relationships between climatic variables at different temporal and spatial scales.

How to cite: Duarte, S., Corzo, G., and Santos, G.: Identification of Synchronicity in Deterministic Chaotic Attractors for the Downscaling Process in the Bogotá River Basin, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14752, https://doi.org/10.5194/egusphere-egu21-14752, 2021.

Groundwater pollution numerical simulations coupled with Genetic Algorithms (GAs) lead to vast computational load, while flow fields’ simplification can compensate in design, but not real-time/operational, applications. Various Machine Learning/Deep Learning (ML/DL) methods/problem-formulations were tested/evaluated for real-time inverse problems of aquifer pollution source identification. Aim: investigate data-driven approaches towards replacing flow simulation with ML/DL trained models identifying the source, faster but efficiently enough.

Steady flow in a 1500mx1500m theoretical confined, isotropic aquifer of known characteristics is studied. Two pumping wells (PWs) near the southern boundary provide irrigation/drinking water, defining the flow together with a varying North-South natural flow. Six suspected possible sources, capable of instantaneous leakage, may spread a conservative pollutant. Particle tracking simulates advective mass transport, in a 2D flow-field for 2500 1-day timesteps. The 14x14 inner field grid nodes serve as locations of sources, PWs and monitoring wells (MWs; for simple daily yes/no pollution detection and/or drawdown measuring). 15,246 combinations of 6 Source Nrs, 21 N-S hydraulic gradients, 11+11 PW1,2 flow-rates were simulated with existing own software, providing the necessary data-sets for ML training/evaluation.

Two basic ML/DL approaches were implemented: Classification (CL) and Computer Vision (CV). In CL, every source is a discrete class, while each MW is a discrete variable. The target variable Y can equal 1 to 6, while input variables X can be: a) 0/1 (MW_i polluted or not), b) the first day of MW_i’s pollution, c) the duration of MW_i’s pollution, d) hydraulic drawdown of MW_i. For a bit more realism, the two southern rows of 28 MWs, and the MWs on/around PWs are concealed. CL features the advantage of facilitating Correlation-based Feature Subset Selection (CFSS), indirectly leading to a pseudo-optimization of the monitoring network, minimizing the number of MWs (not the sampling frequency though), based solely on the efficiency in identifying the source criterion. As a downside, time dimension and spatial correlation of MWs are not considered. Approach (b) being the best scheme, Random Forests (RFs; 86.5576% accuracy), Multi-Layer Perceptron (MLP; 77.5%), and Nearest Neighbors (NN; 86.5%) were tested. CFSS led to 8 only MWs being important, so training with the optimal subsets gave promising results: RF=85.4%, MLP=73.1%, NN=85.4%. In CV, MW_is’ pollution input data on a 10-day basis (0-60, 800-on concealed) were formulated into 14x14-pixel black/white images, that is 14x14 binary (0,1) matrices, the t=0 image being the desideratum. A Convolutional Neural Network (CNN; U-Net architecture for image segmentation) achieved 97.1% accuracy. A Convolutional Long/Short-Term Memory Neural Network (CLSTM), training a model to back-propagate predicting each given time step, with unchanged data formulation (60-800d, step 10), exhibits 82.3% accuracy. CLSTM’s performance is timestep-sensitive, best results yielded (98% accuracy) using configuration 5-800d, step 6.

Concluding, CL’s CFSS minimizes the input space, while CV approaches yield more promising results in terms of accuracy. Each approach has certain constraints in operational applicability, concerning the number of MWs, the sampling resolution and the total elapsed time. This process paves the way for realistic inverse problem solutions, ML-GAs monitoring network optimization, and real-time pollution detection operational systems. 

How to cite: Kontos, Y., Kassandros, T., Katsifarakis, K., and Karatzas, K.: Groundwater pollution monitoring and the inverse problem of source identification. Evaluation of various Machine Learning methods, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-16388, https://doi.org/10.5194/egusphere-egu21-16388, 2021.

Hydrological calibrations with historical data are often deemed insufficient for deducing safe estimations about a model structure that imitates, as closely as possible, the anticipated catchment behaviour. Ιn order to address this issue, we investigate a promising strategy, using as drivers synthetic time series, which preserve the probabilistic properties and dependence structure of the observed data. The key idea is calibrating a model on the basis of synthetic rainfall-runoff data, and validating against the full observed data sample. To this aim, we employed a proof of concept on few representative catchments, by testing several lumped conceptual hydrological models with alternative parameterizations and across two time-scales, monthly and daily. Next, we attempted to reinforce the validity of the recommended methodology by employing monthly stochastic calibrations in 100 MOPEX catchments. As before, a number of different hydrological models were used, for the purpose of proving that calibration with stochastic inputs is independent of the chosen model. The results highlight that in most cases the new approach leads to stronger parameter identifiability and stable predictive capacity across different temporal windows, since the model is trained over much extended hydroclimatic conditions.

How to cite: Kourakos, V., Efstratiadis, A., and Tsoukalas, I.: Can hydrological model identifiability be improved? Stress-testing the concept of stochastic calibration, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-11704, https://doi.org/10.5194/egusphere-egu21-11704, 2021.

Hydrological forecasts at different horizons are often made using different models. These forecasts are usually temporally inconsistent (e.g., monthly forecasts may not sum to yearly forecasts), which may lead to misaligned or conflicting decisions. Temporal hierarchal reconciliation (or simply, hierarchical reconciliation) methods can be used for obtaining consistent forecasts at different horizons. However, their effectiveness in the field of hydrology has not yet been investigated. Thus, this research assesses hierarchal reconciliation for precipitation forecasting due to its high importance in hydrological applications (e.g., reservoir operations, irrigation, drought and flood forecasting). Original precipitation forecasts (ORF) were produced using three different models, including ‘automatic’ Exponential Time-Series Smoothing (ETS), Artificial Neural Networks (ANN), and Seasonal Auto-Regressive Integrated Moving Average (SARIMA). The forecasts were produced at six timescales, namely, monthly, 2-monthly, quarterly, 4-monthly, bi-annual, and annual, for 84 basins selected from the Canadian model parameter experiment (CANOPEX) dataset. Hierarchical reconciliation methods including Hierarchical Least Squares (HLS), Weighted Least Squares (WLS), and Ordinary Least Squares (OLS) along with the Bottom-Up (BU) method were applied to obtain consistent forecasts at all timescales.

Generally, ETS and ANN showed the best and worst performance, respectively, according to a wide range of performance metrics (root mean square error (RMSE), normalized RMSE (nRMSE), mean absolute error (MAE), normalized MAE (nMAE), and Nash-Sutcliffe Efficiency index (NSE)). The results indicated that hierarchal reconciliation has a dissimilar impact on the ORFs’ accuracy in different basins and timescales, improving the RMSE in some cases while decreasing it in others. Also, it was highlighted that for different forecast models, hierarchical reconciliation methods showed different levels of performance. According to the RMSE and MAE, the BU method outperformed the hierarchical methods for ETS forecasts, while for ANN and SARIMA forecasts, HLS and OLS improved the forecasts more substantially, respectively. The sensitivity of ORF to hierarchical reconciliation was assessed using the RMSE. It was shown that both accurate and inaccurate ORF could be improved through hierarchical reconciliation; in particular, the effectiveness of hierarchical reconciliation appears to be more dependent on the ORF accuracy than it is on the type of hierarchical reconciliation method.

While in the present work, the effectiveness of hierarchical reconciliation for hydrological forecasting was assessed via data-driven models, the methodology can easily be extended to process-based or hybrid (process-based data-driven) models. Further, since hydrological forecasts at different timescales may have different levels of importance to water resources managers and/or policymakers, hierarchical reconciliation can be used to weight the different timescales according to the user’s preference/desired goals.

How to cite: Jahangir, M. S. and Quilty, J.: Improving hydrological forecasts through temporal hierarchal reconciliation, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-13303, https://doi.org/10.5194/egusphere-egu21-13303, 2021.

We present an approach to combine two data-centric approaches, data assimilation (DA) and deep learning (DL), from the perspective of hydrologic forecasting. DA is a statistical approach based on Bayesian filtering to produce optimal states and/or parameters of a dynamic model using observations. By extracting information from both model and observational data, DA improves not only the performance of numerical modeling, but also understanding of uncertainties in predictions. While DA complements information gaps in model and observational data, DL constructs a new modeling system by extracting and abstracting information solely from data without relying on the conventional knowledge of hydrologic systems. In a new approach, an ensemble of deep learning models can be updated by real-time data assimilation when a new observation becomes available. In the presentation, we will focus on discussing the potentials of combining two data-centric approaches.

How to cite: Noh, S. J., Choi, H., and Kim, B.: An ensemble of deep learning models with data assimilation for hydrologic forecasting, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-16249, https://doi.org/10.5194/egusphere-egu21-16249, 2021.

River hydrograph analysis provides valuable information about temporal and spatial variability of the river discharge. One of the most imporant operations is separation of hydrograph, which aims at decomposing the total streamflow into components. Numerous approaches for hydrograph separation have been developed to date. Most of them traditionally separate the streamflow into general quickflow and baseflow components, but it is also possible to obtain more specific quickflow separation with subdivision into genetic components, such as seasonal snowmelt, rain, thaw etc. We present the general framework for river hydrograph analysis and separation provided by newly released GrWat package, which has been developed during several years. The framework includes a simple tabular data model for representation of hydrograph and climatic (temperature and precipitation) daily data needed for separation of the quickflow into genetic components; spatial analysis operations for automatic extraction of climatic data from reanalysis datasets covering the river basin; automated interpolation of missing data considering the autocorrelation; fast implementation of multiple algorithms for hydrograph separation; computation of more than 30 interannual and long-term characteristics of separated hydrograph components; scale-space transformation for hierarchical decomposition of the hydrograph; high-quality plotting and reporting of the results of analysis. One of the prominent features of the framework is a powerful algorithm for genetic hydrograph separation, which is capable of not only extracting the baseflow, seasonal, thaw and rain flood components, but also to cut the short-time rain floods which complicate the shape of the seasonal flood. The baseflow separation is performed on the first stage and can be initialized by any of the baseflow separation algorithms available in the package. On the second stage the quickflow is separated into genetic components. Such modular structure provides the flexible way to experiment with different combinations of algorithms and to select the approach wich serves best to the goal of the analysis and specific features of the hydrograph.

The study was supported by the Russian Science Foundation grant No. 19-77-10032

How to cite: Kireeva, M., Samsonov, T., and Rets, E.: Comprehensive analysis and separation of river hydrograph using the GrWat R package, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14261, https://doi.org/10.5194/egusphere-egu21-14261, 2021.

Satellite based rainfall estimates (SBRE) are used as an alternative to gauge rainfall in hydrological studies particularly for basins with data issues. However, these data products exhibit errors which cannot always be corrected by bias correction methods such as Ratio Bias Correction (RBC). Data fusion or data merging can be a potentially good approach in merging various satellite rainfall products to obtain a fused dataset, which can benefit from all the data sources and may minimise the error in rainfall estimates. Data merging methods which are commonly applied in meteorology and hydrology are: Arithmetic merging method (AMM), Inverse error squared weighting (IESW) and Error variance (EV). Among these methods EV is popular, which merges can be used to merge bias corrected SBREs using the minimisation of variance principle.

In this research we investigated the possibility of using K nearest neighbour as a data merging method. Four satellite rainfall products were used in this study namely CMORPH, PERSIANN CDR, TRMM 3B42 and MSWEP. MSWEP was used as a reference dataset for comparing the merged rainfall dataset since it is also a merged product. All these products were downloaded at 0.25° x 0.25° spatial scale and daily temporal scale. Satellite products are known to behave differently at different temporal and spatial scales. Based on the climatic and physiographic features the Indus basin was divided into four zones. Rainfall products were compared at daily, weekly, fortnightly, monthly and seasonal scales whereas spatial scales were gauge location, zonal scales and basin scale. The RBC method was used to correct the biasness of satellite products by correcting the products at monthly and seasonal scale. Wth bias correction the daily normalised root mean square error (NRMSE) was reduced up to 20% for CMORPH, 22% for PERSIANN CDR and 14% for TRMM at the Indus basin scale for monthly scale which is why the monthly bias corrected data was used for merging. Merging of satellite products can be fruitful to benefit from the strength of each product and minimize the weakness of products. Four different merging methods i.e. Arithmetic merging method (AMM), Inverse error squared weighting (IESW), Error variance (EV) and K Nearest Neighbour method (KNN) were used and performance was checked in two seasons i.e. non-wet and wet season. AMM and EV methods performed similarly whereas IESW performed poorly at zonal scales. KNN merging method outperformed all other merging methods and gave lowest error across the basin. Daily NRMSE was reduced to 0.3 at Indus basin scale with KNN method, AMM and EV reduced the error to 0.45 in comparison to error produced by CMORPH, PERSIANN CDR and TRMM of 0.8, 0.65 and 0.53 respectively in the wet season. KNN merged product gave lowest error at daily scale in calibration and validation period which justifies that merging improves rainfall estimates in sparsely gauged basin.

Key words: Merging, data fusion, K nearest neighbour, KNN, error variance, Indus.

How to cite: Bhattacharya, B. and Ahmad, J.: K nearest neighbour in merging satellite rainfall estimates from diverse sources in sparsely gauged basins, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14650, https://doi.org/10.5194/egusphere-egu21-14650, 2021.

Due to the importance of object detection in video analysis and image annotation, it is widely utilized in a number of computer vision tasks such as face recognition, autonomous vehicles, activity recognition, tracking objects and identity verification. Object detection does not only involve classification and identification of objects within images, but also involves localizing and tracing the objects by creating bounding boxes around the objects and labelling them with their respective prediction scores. Here, we leverage and discuss how connected vision systems can be used to embed cameras, image processing, Edge Artificial Intelligence (AI), and data connectivity capabilities for flood label detection. We favored the engineering definition of label detection that a label is a sequence of discrete measurable observations obtained using a capturing device such as web cameras, smart phone, etc. We built a Big Data service of around 1000 images (image annotation service) including the image geolocation information from various flooding events in the Carolinas (USA) with a total of eight different object categories. Our developed platform has several smart AI tools and task configurations that can detect objects’ edges or contours which can be manually adjusted with a threshold setting so as to best segment the image. The tool has the ability to train the dataset and predict the labels for large scale datasets which can be used as an object detector to drastically reduce the amount of time spent per object particularly for real-time image-based flood forecasting.  This research is funded by the US National Science Foundation (NSF).

How to cite: Pally, J. R. R. and Samadi, V.: Application of Image Processing and Big Data Science for Flood Label Detection, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-3709, https://doi.org/10.5194/egusphere-egu21-3709, 2021.

The assessment of TSM spatiotemporal variability plays a key role in inland water management, considering how these fluctuations affect water transparency, light availability, and the physical, chemical, and biological processes. All the above-mentioned topics highlight the need to develop innovative methodologies of data analysis that are able to handle multi-mission and multi-source remote sensing data, fostering the implementation of integrated and sustainable approaches. Sentinel-2A multispectral instrument (MSI) and Landsat 8 operational land instrument (OLI) data offer unique opportunities for investigating certain in-water constituents (e.g., TSM and chlorophyll-a) mainly owing to their spatial resolution (10–60 m). Furthermore, the joint use of these sensors offers the opportunity to build time series with an improved revisiting time thus enabling limnologists, aquatic ecologists and water resource managers to enhance their monitoring efforts. In this framework, the potential of MSI–OLI combined data in characterizing the multi-temporal (2014–2018) TSM variability in Pertusillo Lake (Basilicata region, Southern Italy) has been evaluated in this work. In particular, a customized MSI-based TSM model (R²=0.81) has been developed and validated by using ground truth data acquired during specific measurement campaigns. The model was then exported on OLI data through an inter-calibration procedure (R²=0.87), allowing for the generation of a TSM multi-temporal MSI–OLI merged dataset. The analysis of the derived multi-year TSM monthly maps has shown the influence of hydrological factors on the TSM seasonal dynamics over two sub-regions of the lake, the west and east areas. The western side appears more affected by inflowing rivers and water level fluctuations, whose  effects  tend to longitudinally decrease, leading to less sediment within the eastern sub-area. The achieved results highlight how the proposed methodological approach (i.e. in situ data collection, satellite data processing and modeling) can be exported in other inland waters that deserve to be investigated for a better management of water quality and monitoring systems.

How to cite: Ciancia, E., Campanelli, A., Lacava, T., Palombo, A., Pascucci, S., Pergola, N., Pignatti, S., Satriano, V., and Tramutoli, V.: On the potential of multi-source remote sensing data in characterizing the Total Suspended Matter variability in inland waters., EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14711, https://doi.org/10.5194/egusphere-egu21-14711, 2021.

Urban water systems are designed with the goal of delivering their service for several decades.  The infrastructure will inevitably face long-term uncertainty in a multitude of parameters from the hydroclimatic and socioeconomic realms (e.g., climate change, limited supply of water in terms quantity and acceptable quality, population growth, shifting demand patterns, industrialization), as well as from the conceptual realm of the decision maker (e.g., changes in policy, system maintenance incentives, investment rate, expansion plans). Because urban water systems are overly complex, a holistic analysis involves the use of various models that individually pertain to a smaller sub-system and a variety of metrics to assess performance, whereas the analysis is accomplished at different temporal and spatial scales for each sub-system. In this work, we integrate a water resources management model with a water distribution model and a water demand generation model at smaller (household and district) scale, allowing us to simulate urban water systems “from source to tap”, covering the entire water cycle. We also couple a stochastic simulation module that supports the representation of uncertainty throughout the water cycle. The performance of the integrated system under long term uncertainty is assessed with the novel measure of system’s resilience i.e. the degree to which a water system continues to perform under progressively increasing disturbance. This evaluation is essentially a framework of systematic stress-testing, where the disturbance is described via stochastically changing parameters in an ensemble of scenarios that represent future world views. The framework is showcased through a synthesized case study of a medium-sized urban water system.

Acknowledgement

This research is carried out / funded in the context of the project “A resilience assessment framework for water supply infrastructure under long-term uncertainty: A Source-to-Tap methodology integrating state of the art computational tools” (MIS 5049174) under the call for proposals “Researchers' support with an emphasis on young researchers- 2nd Cycle”. The project is co-financed by Greece and the European Union (European Social Fund- ESF) by the Operational Programme Human Resources Development, Education and Lifelong Learning 2014-2020.”

How to cite: Nikolopoulos, D., Kossieris, P., and Makropoulos, C.: Stochastic stress-testing approach for assessing resilience of urban water systems from source to tap, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-13284, https://doi.org/10.5194/egusphere-egu21-13284, 2021.

The issue of vulnerability and robustness in networked systems can be addressed by several methods. The most widely used are based on a set of centrality and connectivity measures from network theory which basically relate vulnerability to the loss of efficiency caused by the removal of some nodes and edges. Another related view is given by the analysis of the spectra of the adjacency and Laplacian matrices of the graph associated to the networked system.

The main contribution of this paper is the introduction of a new set of vulnerability metrics given by the distance between the probability distribution of node-node distances between the original network and that resulting from the removal of nodes/edges. Two such probabilistic measures have been analysed: Jensen-Shannon (JS) divergence and Wasserstein (WST) distance, aka the Earth-Mover distance: this name comes from its informal interpretation as the minimum energy cost of moving and transforming a pile of dirt in the shape of one probability distribution to the shape of the other distribution. The cost is quantified by the amount of dirt moved times the moving distance. The Wasserstein distance can be traced back to the works of Gaspard Monge in 1761 and Lev Kantorovich in 1942. Wasserstein distances are generally well defined and provide an interpretable distance metric between distributions. Computing Wasserstein distances requires in general the solution of a constrained linear optimization problem which is, when the support of the probability distributions is multidimensional, very large.

An advantage of the Wasserstein distance is that, under quite general conditions, it is a differentiable function of the parameters of the distributions which makes possible its use to assess the sensitivity of the network robustness to distributional perturbations. The computational results related to two real-life water distribution networks confirm that the value of the distances JS and WST is strongly related to the criticality of the removed edges. Both are more discriminating, at least for water distribution networks, than efficiency-based and spectral measures. A general methodological scheme has been developed connecting different modelling and computational elements, concepts and analysis tools, to create an analysis framework suitable for analysing robustness. This modelling and algorithmic framework can also support the analysis of other networked infrastructures among which power grids, gas distribution and transit networks.

How to cite: Ponti, A., Candelieri, A., and Archetti, F.: Vulnerability and robustness of networked infrastructures: beyond typical graph-based measures, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-12708, https://doi.org/10.5194/egusphere-egu21-12708, 2021.

Floods are among the most destructive natural hazard that affect millions of people across the world leading to severe loss of life and damage to property, critical infrastructure, and agriculture. Internet of Things (IoTs), machine learning (ML), and Big Data are exceptionally valuable tools for collecting the catastrophic readiness and countless actionable data. The aim of this presentation is to introduce Flood Analytics Information System (FAIS) as a data gathering and analytics system.  FAIS application is smartly designed to integrate crowd intelligence, ML, and natural language processing of tweets to provide warning with the aim to improve flood situational awareness and risk assessment. FAIS has been Beta tested during major hurricane events in US where successive storms made extensive damage and disruption. The prototype successfully identifies a dynamic set of at-risk locations/communities using the USGS river gauge height readings and geotagged tweets intersected with watershed boundary. The list of prioritized locations can be updated, as the river monitoring system and condition change over time (typically every 15 minutes).  The prototype also performs flood frequency analysis (FFA) using various probability distributions with the associated uncertainty estimation to assist engineers in designing safe structures. This presentation will discuss about the FAIS functionalities and real-time implementation of the prototype across south and southeast USA. This research is funded by the US National Science Foundation (NSF).

How to cite: Samadi, V. and Pally, R.: The Convergence of IoT, Machine Learning, and Big Data for Advancing Flood Analytics Knowledge , EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-7782, https://doi.org/10.5194/egusphere-egu21-7782, 2021.

Water resources management requires specialized computer tools that allow explicit integration of surface and groundwater fluxes, which generally have domains with different spatial discretization. On one hand, a surface hydrological domain, D1, is typically segmented in sub-basins, elevation contour bands or hydrological response units. These elements usually are represented by grids, triangles, or simple irregular polygons. In D1, the elements are connected to each other and incorporated into a drainage network that defines a surface topology, t1. On the other hand, an aquifer domain, D2, is organized in hydrogeological units, which can be represented by geometrical elements such as grids, triangulations, Voronoi or Quadratree diagrams. In D2, a regular connection is typically associated to structured meshes that defines a groundwater topology, t2. We present a new tool called GeoLinkage (v.geolinkage) that creates an ESRI-format linkage shapefile of the new surface-groundwater topology, t_1-2. This python-based open-source tool has a graphical user interface (GUI) as an add-on for GRASS-GIS, which was constructed using Pygrass and Flopy libraries. It was developed to be used in WEAP-MODFLOV models, but it can also be used with other water resources management models. GeoLinkage allows processing models with reasonable computation times, which facilitates scenario analysis. It calculates the locations of the surface element geometries (nodes and arcs) using the GRASS platform and connects them to each element of a structured mesh in MODFLOW models. GeoLinkage was applied to obtain groundwater levels and coverage of water demand in Azapa Valley, a hyper-arid zone in the desert of Chile, where a grid of 70.305 cells and six fields with detailed geometry were processed in only 12 min.

How to cite: Troncoso, F., Hitschfeld, N., Sanzana, P., Suárez, F., and Muñoz, J.: GeoLinkage: a GRASS-GIS plugin to integrate surface waters and groundwater in WEAP-MODFLOW models, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-13909, https://doi.org/10.5194/egusphere-egu21-13909, 2021.

We demonstrate a framework for urban flood modeling with community mapped data, particularly suited for flood risk management in data-scarce environments. The framework comprises three principal stages: data acquisition with survey design and quality assurance, model development and model implementation for flood prediction. We demonstrate that data acquisition based on community mapping can be affordable, comprehensible, quality assured and open source, making it applicable in resource-strained contexts. The framework was demonstrated and validated on a case study in Dar es Salaam, Tanzania. The results obtained show that the community mapped data supports flood modeling on a level of detail that is currently inaccessible in many parts of the world. The results obtained also show that the community mapping approach is appropriate for datasets that do not require extensive training, such as flood extent surveys where it is possible to cross-validate the quality of reports given a suitable number and density of data points. More technically advanced features such as dimensions of urban drainage system elements still require trained mappers to create data of sufficient quality. This type of mapping can, however, now be performed in new contexts thanks to the development of smartphones. Future research is suggested to explore how community mapping can become an institutionalized practice to fill in important gaps in data-scarce environments.

How to cite: Petersson, L., ten Veldhuis, M.-C., Verhoeven, G., Kapelan, Z., Maholi, I., and Winsemius, H.: Community Mapping Supports Comprehensive Urban Flood Modeling for Flood Risk Management in a Data-Scarce Environment, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4221, https://doi.org/10.5194/egusphere-egu21-4221, 2021.

We present the first trial of an accurate, low-cost wireless sensor, the ‘Hydrobean’, and base station designed for use by citizen scientists in catchment water quality monitoring. This novel wireless sensor network addresses key concerns identified with current volunteer monitoring programmes, including temporal discontinuity and insufficient data quality. Hydrobean continuously measures electrical conductivity, temperature and pressure and wirelessly transmits these data to an online portal for observation and download by users. These parameters can be used to assess catchment water quality status, with excursions from baseline conditions detected in real time at high temporal resolution. Citizen scientists have an increasingly important role to play in enhancing our scientific understanding of catchment water quality, but their contribution has so far been limited by barriers to access suitable monitoring equipment. Traditional grab sampling techniques result in key contamination incidents being missed and trend analysis limited as samples are analysed discretely, typically on a weekly or monthly basis. Additionally, the quality of data obtained from basic chemical test kits commonly used by citizen scientists does not meet the requirements of many data users. This research explores the role of low-cost wireless sensor networks in advancing the potential of citizen scientists in monitoring catchment water quality. Monitoring equipment available to citizen scientists generally needs to be low cost, so is unlikely to rival professional standard monitoring techniques in the foreseeable future. However, reliable, low-cost sensors which enable continuous, real-time monitoring do now exist for a limited range of water quality parameters and have been used in the development of the wireless sensor network presented here. Critically, Hydrobean and its base station are low cost, low maintenance, portable and robust in order to meet the requirements of community monitoring programmes. Ultimately, a model will be integrated into the real-time analysis of data collected by the wireless sensor network to predict when and where contamination incidents are expected to be affecting catchment water quality. We report initial field results of the Hydrobean wireless sensor network and will discuss ways in which the basic design can be improved in future versions. 

How to cite: von Benzon, E., Bagshaw, E., Prior-Jones, M., Sobey, I., Perkins, R., and Browning, S.: A low-cost wireless sensor network for citizen science water quality monitoring, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-12426, https://doi.org/10.5194/egusphere-egu21-12426, 2021.

Crowd-sourcing of hydrological data with volunteering citizen scientists has the potential to overcome severe data limitations in space and time. However, several aspects on the reliability, quality, and value of crowd-sourced data are under debate. In this contribution, we present results of a citizen science experiments involving 300 high school students in Luxembourg. The students relied on self-build rainfall collectors to sample precipitation over selected 24-hour periods covering Luxembourg at national scale (~2500 km²) and subsequently measured the amount. Following data collection and archiving, we evaluated the quality of the data by benchmarking the crowd-sourced values to data collected with a dense network of ~50 tipping buckets across the country. This was done by kriging both data sets. We found that the aerial precipitation at national scale derived from both data sort was consistent, however with a rather systematic bias between the two data sources. The bias was in the same range as the bias between tipping bucket data and average amounts from several measurements with a self-build sampler at the same location. The students’ data showed a clearly higher variance compared to the national data but was still able to resolve finer scale variations compared to the national network. We observed the highest differences between both data sets in urban settings. Here, it is not clear if the student’s data was less robust when acquired in an urban setting or if the difference arose from urban rainfall processes that were not observed by the national network, where stations are placed at open sites. With our proposed experiment and the statistical data analysis, we were able to quality control crowd-sourced precipitation data and showed that they are reliable. This increases confidence for many studies relying on similar samplers. Yet, some samples of individuals showed a rather high deviation from the kriged national network, showing that sampling with only a few citizen observers could lead to higher uncertainty in the data. While some limitations exist, we showed that data from citizens are of high quality and provide valuable information for hydrological studies.

How to cite: Klaus, J., Hannah, D., and Chun, K. P.: How reliable are crowd-sourced data in hydrology? Lessons learned from a citizen science experiment, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4825, https://doi.org/10.5194/egusphere-egu21-4825, 2021.