EGU23-11187, updated on 10 Jan 2024
https://doi.org/10.5194/egusphere-egu23-11187
EGU General Assembly 2023
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.

Fostering collaboration through improved software development practices for the ONEFlux eddy covariance data processing pipeline

Gilberto Pastorello1, Carlo Trotta2,3, Alessio Ribeca3, Keith Beattie1, Sy-Toan Ngo1, Housen Chu1, You-Wei Cheah1, Danielle Christianson1, Giacomo Nicolini2,3, Sigrid Dengel1, Diego Polidori3, Peter Isaac4, Matthew Archer5, Dominic Orchard5, Deb Agarwal1, Sebastien Biraud1, Margaret Torn1,6, and Dario Papale3
Gilberto Pastorello et al.
  • 1Lawrence Berkeley National Laboratory, Berkeley, United States of America
  • 2Euro-Mediterranean Centre on Climate Change (CMCC), Viterbo, Italy
  • 3University of Tuscia, Viterbo, Italy
  • 4TERN-OzFlux, Melbourne, Australia
  • 5University of Cambridge, Cambridge, United Kingdom
  • 6University of California, Berkeley, United States of America

Standardized processing of eddy covariance data is important for studies combining data from multiple sites, for validating remote sensing measurements as well as runs of ecosystem and climate models, and for applications relying on these flux data to create derived products like upscaled fluxes, among other examples. However, maintaining consistency within the software used for this processing while allowing for evolution of this code across research networks presents novel challenges in software development. The introduction of the ONEFlux (Open Network-Enabled Flux) eddy covariance data processing pipeline, originally developed within a collaboration of the AmeriFlux Management Project, the European Fluxes Database, and the ICOS Ecosystem Thematic Centre, supported the creation of consistently processed global eddy covariance data products. In particular, ONEFlux codes were used to generate the FLUXNET2015 dataset, which is widely adopted by thousands of eddy covariance data users in their work in research, ranging from soil microbiology to large scale drought effects, and also education, from basic plant biology all the way to global climate change. We are now more thoroughly instrumenting the code, and the code development process, to better address these challenges, efforts which we will describe in this presentation. In particular, we are seeking to improve software development practices to allow for more streamlined collaboration on expanding and contributing to the codebase. For instance, we are adopting planned release cycles for code updates, designing more detailed ways to incorporate and evaluate new modules, introducing data-centric testing and continuous integration, improving code performance, and adopting several other software engineering best practices more widely in the development workflows. The main goal of these changes is to lower the barriers for running ONEFlux by regional networks processing their data, while at the same time better supporting contributions from the community into the codebase. This will be critical to continue the current use of ONEFlux to generate updated versions of flux datasets by regional networks, the components of new global products.

How to cite: Pastorello, G., Trotta, C., Ribeca, A., Beattie, K., Ngo, S.-T., Chu, H., Cheah, Y.-W., Christianson, D., Nicolini, G., Dengel, S., Polidori, D., Isaac, P., Archer, M., Orchard, D., Agarwal, D., Biraud, S., Torn, M., and Papale, D.: Fostering collaboration through improved software development practices for the ONEFlux eddy covariance data processing pipeline, EGU General Assembly 2023, Vienna, Austria, 23–28 Apr 2023, EGU23-11187, https://doi.org/10.5194/egusphere-egu23-11187, 2023.