The International Soil Moisture Network (ISMN): revised flagging strategy and AI assisted quality control

Soil moisture is a key variable influencing land–atmosphere interactions, hydrological extremes, ecosystem processes, and agricultural productivity. The International Soil Moisture Network (ISMN) provides a global, freely-accessible repository of quality-controlled in situ soil moisture observations to support Earth system science, remote sensing validation, and model development through standardized and traceable data. The ISMN compiles soil moisture time series from a wide range of regional, national, and international monitoring networks. Contributing datasets are harmonized in terms of format, metadata, and temporal resolution and subjected to a uniform, rule-based quality control (QC) procedure to ensure research-ready data.

Each observational data point undergoes thirteen plausibility checks, resulting in flagging data as “good” or “dubious”. These checks fall into three categories: (i) a geophysical range verification, identifying  thresholds exceedances (e.g., soil moisture < 0% Vol); (ii) geophysical consistency checks, comparing observations with ancillary in situ data or NASA’s GLDAS Noah model data (e.g., flagging of soil moisture when soil temperature is below 0°C); and (iii) spectrum-based approaches, using the first and second derivatives of soil moisture timeseries to detect irregular patterns such as spikes, breaks, or plateaus.

In this work, we propose targeted adaptations to the existing QC flagging strategy to reduce false positives, where valid measurements are incorrectly marked as “dubious”. These refinements increase the proportion of data points flagged as “good” by up to 15% for the entire database. Also, we are proposing the revision of several flags which are originally optimized for the validation of remote sensing products to enhance usability across broader scientific applications, while still maintaining their utility for the remote sensing community. Finally, we will introduce an AI based change detection algorithm designed to identify and potentially homogenize structural breaks and impute missing or “dubious” values in soil moisture timeseries, such as those caused by sensor replacements. This would enable the generation of longer, more consistent time series records suitable for statistically robust trend analyses.