An automated machine learning framework for multi-depth soil moisture prediction using hydro-meteorological datasets

 Soil moisture, one of the essential climate variables, forms a fundamental bridge between hydro-meteorological processes and influence climate dynamics. It is extremely variable and is driven by numerous hydrological, agricultural and ecological factors. Soil moisture subsequently impacts soil forming processes, root zone water availability, infiltration rates, runoff, groundwater storage and vegetation-soil interaction. Despite its significant contribution in hydro-ecological interaction, its variability at subsurface is not yet explored adequately. Precise estimation of soil moisture at various depths is crucial because it affects water retention characteristics and modulates the vertical and lateral movement of water within the soil profile. This subsurface information is integral to understanding recharge rates, groundwater interactions, and the overall water balance within a catchment. In this study, we present an automated machine learning framework designed to predict soil moisture at multiple depths of 10 cm, 20 cm, 30 cm, and 40 cm leveraging Bayesian optimization. We collected data from our hydrological observatory set up constituting an automatic weather station, a pan evaporimeter and a soil moisture recorder. To evaluate model performance, we categorized the dataset into four scenarios (S1, S2, S3, and S4), with each subsequent scenario incorporating a greater number of observations and rainfall events. We used 11 input features to train this AutoML model by integrating several hydrological and meteorological variables with in-situ soil moisture data. Among the predictor variables, humidity, dew point, and rainfall emerged as the most influential factors driving soil moisture variability. The model was trained to calculate the performance metrices for the entire dataset and for subsets containing only rainfall instances. Our optimized model demonstrated superior performance, with an R² of 0.88–0.99 and RMSE < 0.022 for the overall dataset, and R² of 0.76–1.00 with RMSE < 0.06 for rainfall-specific data across all soil moisture depths.