CCP: A Cloud Computing Platform for VREs in Earth Sciences

The Cloud Computing Platform (CCP) was developed under the aegis of D4Science [1]. D4Science is an operational digital infrastructure co-funded by the European Commission, and represents a significant advancement in supporting the FAIR (Findable, Accessible, Interoperable, and Reusable) principles, open science, and reproducible data-intensive science. D4Science has evolved to harness the "as a Service" paradigm, offering web-accessible Virtual Research Environments (VREs) [2] that have also been instrumental in facilitating science collaborations [3] with a particular focus on Earth observation, Earth science, and marine and agricultural environments. These environments simplify access to datasets while concealing underlying complexities, and include functionalities such as a cloud-based workspace for file organisation, a platform for large-scale data analysis, a catalogue for publishing research results, and a communication system rooted in social networking practices.

A key component for enabling large-scale, affordable, and reproducible computation and data analysis is CCP: a cloud computing platform specifically designed for VREs and Open Science.

CCP enables researchers to import, execute, and share methods ranging from statistical analysis to image classification, from AI models to 3D reconstruction, from data format conversion to pattern searching in DNA sequences, while embodying FAIR (Findable, Accessible, Interoperable, and Reusable) principles.

By leveraging container technology, an API-based design, and adherence to standards such as the OGC Processes API [4], CCP supports high interoperability, flexibility and integrability in scientific workflows. Methods can be written in any programming language (Python, Julia, R, etc) and executed either via dedicated web UIs or programmatically from virtually any development environment (command line, custom applications, Galaxy workflows, Jupyter notebooks, RStudio, etc). Code generators are provided to ease the integration into common scientific tools.

CCP can be deployed on container orchestration platforms, such as Docker Swarm or Kubernetes, which can leverage specialized hardware configurations (e.g., HPC clusters or GPU-enabled nodes) depending on the policies and resources available, thereby offering flexible and scalable computational environments per the needs of each community.

Automatic provenance management captures the complete history of a method's execution for reproducibility and accountability, according to common provenance models (Prov-O, RO-crate).  Re-submitting executions can be as simple as clicking on a shared link.

CCP has been integrated into several VREs, many related to Earth science including the Blue-Cloud [5] virtual laboratories and demonstrators and ITINERIS [6].

Keywords:

Virtual Research Environments, Open Science, Cloud Computing, FAIR, Provenance Management, Data Analysis

 

Acknowledgements:

This work is supported by the European Community’s HEU Program under the scheme ‘HORIZON-INFRA-2022-EOSC-01’, grant agreement #101094227 ‘Blue-Cloud 2026: A federated European FAIR and Open Research Ecosystem for oceans, seas, coastal and inland waters [5]’

 

References

1. M. Assante et al. (2019) “Enacting open science by D4Science”. Future Gener. Comput. Syst. 101: 555-563 10.1016/j.future.2019.05.063

2. L. Candela, D. Castelli and P. Pagano(2023) “The D4Science Experience on Virtual Research Environments Development". in IEEE Computing in Science & Engineering, doi: 10.1109/MCSE.2023.3290433

3. M. Assante et al. (2023) “Virtual research environments co-creation: The D4Science experience”. Concurrency Computat Pract Exper. 2023; 35(18):e6925. doi:10.1002/cpe.6925

4. https://ogcapi.ogc.org/processes/

5. https://cordis.europa.eu/project/id/101094227

6. ITINERIS, Italian Integrated Environmental Research Infrastructures System, Funded by EU - Next Generation EU PNRR- Mission 4 “Education and Research” https://itineris.cnr.it/