EGU22-6332
https://doi.org/10.5194/egusphere-egu22-6332
EGU General Assembly 2022
© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

Implementation of a FAIR Compliant Automated Workflow for Infrastructures

Ulrich Bundke, Marcel Kennert, Christoph Mahnke, Susanne Rohs, and Andreas Petzold
Ulrich Bundke et al.
  • Forschungszentrum Jülich GmbH, IEK8 Globale Beobachtungen, Jülich, Germany (u.bundke@fz-juelich.de)

The European infrastructure In-service Aircraft for a Global Observing System (IAGOS) (www.IAOGS.org) has implemented an automatic workflow for data management organizing the dataflow starting at the sensor towards the central data-portal located in Toulouse. The workflow is realized and documented using the web-based Django framework with a model-based approach using Python.

This workflow performs all necessary data processing and QA/QC tests to automated upload NRT processed data and serves the PI as basis for approval decisions. This includes repeated cycles for different stages of data maturity. The PI can monitor the status of all tasks by web-based reports produced by the Task Manager.  An automated reprocessing is possible by storing metadata on all steps as well as decisions of the PI. The implementation of the workflow is one big step to make IAGOS data handling compliant with the FAIR principles (findable, accessible, interoperable, reusable).

The workflow is easy adaptable to manage the workflow of other Infrastructures or research institutes. Thus, we will open the development under MIT license and invite other datacenters to contribute to the development.

Acknowledgments:

This work was supported by European Union's Horizon 2020 research and innovation programme under grant agreement No 824068 and by Helmholtz STSM Grant “DIGITAL EARTH”

How to cite: Bundke, U., Kennert, M., Mahnke, C., Rohs, S., and Petzold, A.: Implementation of a FAIR Compliant Automated Workflow for Infrastructures, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-6332, https://doi.org/10.5194/egusphere-egu22-6332, 2022.