EGU24-18749, updated on 11 Mar 2024
EGU General Assembly 2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.

Tackling practical challenges in anomaly detection for real-time monitoring of urban waste water networks

Lennart Schmidt1, Felix Weiske2, Manfred Schütze3, Phillip Grimm2, Julius Polz4, and Jan Bumberger1
Lennart Schmidt et al.
  • 1Department for Monitoring and Exploration Technologies/Reseach Data Management, Helmholtz Centre for Environmental Research GmbH (UFZ), Leipzig, Germany (
  • 2Grimm Water Solutions UG, Freiburg, Germany
  • 3Institute for Automation and Communication, Magdeburg, Germany
  • 4Institute of Meteorology and Climate Research, Karlsruhe Institute of Technology (Campus Alpin), Garmisch-Partenkirchen, Germany

Waste water networks constitute a crucial element of urban infrastructure that are influenced by an observed increase in urban flooding events. To ensure regular network operation and minimal environmental impact, anomaly detection of urban waste water networks timeseries can serve as a real-time monitoring tool to detect a) sensor defects and b) system anomalies such as leaks or blockages. However, setting up such a monitoring system in practice can face significant challenges. These include limited amounts of labeled anomalies, heterogenous data quality, inconsistent measurement frequencies as well as instationarity of the system (sensor displacement and drop-out, changes in network layout). For the waste water network of a medium-sized German city, we set up machine learning based anomaly detection and present strategies to tackle aforementioned challenges. Our results show that autoencoder-based model architectures are valuable tools in such a context where only a minimal fraction (<0.01%) of the data is labeled. Both a well-parametrized interpolation strategy and a model architecture that is largely robust to missing values are essential prerequisites for adequate model performance. Based on our results, we derive general strategies to aid in setting up anomaly detection systems in real-world use cases.

How to cite: Schmidt, L., Weiske, F., Schütze, M., Grimm, P., Polz, J., and Bumberger, J.: Tackling practical challenges in anomaly detection for real-time monitoring of urban waste water networks, EGU General Assembly 2024, Vienna, Austria, 14–19 Apr 2024, EGU24-18749,, 2024.