Digital Ecosystem for Time Series Data Management in Earth System Science

Understanding and managing the Earth System requires sustainable, interdisciplinary approaches to data accessibility, integration, and processing. To address these challenges, we present a modular and scalable digital ecosystem designed to enhance earth data science and support multidisciplinary applications [1]. Adhering to the FAIR data as well as the FAIR research software principles, the system employs standardized interfaces, and open-source technologies to foster collaboration across disciplines, extending beyond Earth System Sciences.

The ecosystem comprises three core components: (i) the Sensor Management System (SMS) for detailed metadata registration and management [2]; (ii) time.IO, a platform for efficient storage, transfer, and real-time visualization of time series data [3]; and (iii) the System for Automated Quality Control (SaQC), which ensures data integrity through automated data analysis and quality assurance [4,5]. Developed, maintained, and distributed as dedicated projects, these components integrate seamlessly into a coherent time series data management system. Leveraging widely adopted solutions and standards such as the OGC SensorThings API, OGC SensorML and the EUDAT B2INST persistent identifier, the system ensures compatibility and integration across research infrastructures, software systems, and diverse disciplines.

This cloud-ready and highly adaptable ecosystem supports deployments from small-scale local research projects to large-scale international environmental monitoring networks. It provides a user centric solution for storing, analyzing, and visualizing data. The use of established metadata standards and the community-driven development of metadata schemes and semantic annotations ensure consistency, interoperability, and reusability of metadata and data formats across various applications The applicability of the proposed ecosystem for use cases from Earth System Sciences and its usability across all stages of a typical sensor data lifecycle will be demonstrated using Cosmic Ray Neutron Sensing data as an illustrative example.

By aligning user needs with sustainable software solutions, this ecosystem facilitates FAIR-compliant practices, supports scientific innovation, and promotes robust, transparent research in Earth System sciences.

 

References:

[1] Bumberger, J., Abbrent, M., Brinckmann N., Hemmen, J., Kunkel, R., Lorenz, C., Lünenschloß, P., Palm, B., Schnicke, T., Schulz, C., van der Schaaf, H., and Schäfer, D. (2025). Digital Ecosystem for FAIR Time Series Data Management in Environmental System Science. SoftwareX (accepted)

[2] Brinckmann, N., Alhaj Taha, K., Kuhnert, T., Abbrent, M., Becker, W., Bohring, H., Breier, J., Bumberger, J., Ecker, D., Eder, T., Gransee, F., Hanisch, M., Lorenz, C., Moorthy, R., Nendel, L. J., Pongratz, E., Remmler, P., Rosin, V., Schaeffer, M., Schaldach, M., Schäfer, D., Sielaff, D., & Ziegner, N. (2024). Sensor management system - SMS (1.17.1). Zenodo. https://doi.org/10.5281/zenodo.13329925

[3] Schäfer, D., Abbrent, M., Gransee, F., Kuhnert, T., Hemmen, J., Nendel, L., Palm, B., Schaldach, M., Schulz, C., Schnicke, T., & Bumberger, J. (2023). timeIO - A fully integrated and comprehensive timeseries management system (0.1). Zenodo. https://doi.org/10.5281/zenodo.8354839

[4] Schmidt, L., Schäfer, D., Geller, J., Lünenschloss, P., Palm, B., Rinke, K., Rebmann, C., Rode, M., & Bumberger, J. (2023). System for automated Quality Control (SaQC) to enable traceable and reproducible data streams in environmental science. Environmental Modelling & Software, 105809. https://doi.org/10.1016/j.envsoft.2023.105809

[5] Schäfer, D., Palm, B., Lünenschloß, P., Schmidt, L., Schnicke, T., & Bumberger, J. (2024). System for automated Quality Control - SaQC (v2.6.0). Zenodo. https://doi.org/10.5281/zenodo.5888547