Usability in ESS Data Cubes and Cloud Computing: Challenges to Converge in a Landscape of Diverse Solutions

Cloud computing has become essential in Earth System Sciences (ESS) due to the increasing volume of data and the necessity for cross-disciplinary collaboration. Over recent years, this has led to a fragmented landscape of cloud computing solutions, with no clear consolidation emerging. Consequently, users face a diverse range of options, often designed to achieve similar objectives. Despite a strong interest in shifting from traditional workflows to cloud-based processing, user surveys reveal that the transition is hindered by a steep learning curve, significant fragmentation, and particularly, a lack of interoperability among platforms, data, and workflows (Wagemann et al., 2021; DiLeo et al., 2024).

It is already difficult to get an overview of existing solutions in the realm of cloud-based research data infrastructures (RDI) which usually include tools for data providers, platforms, API’s, clients, software and file formats. As a user, finding a way to create meaningful and interoperable end-to-end workflows is even more challenging. Especially, since not all combinations of software, API’s and data are available on all RDIs. Various projects exist with the goal of consolidation of choices in cloud native ESS to increase interoperability and decrease complexity for users, e.g. from the European Space Agency (EOEPCA – Earth Observation Exploitation Platform Common Architecture, APEx – Application Propagation Environment) or the Open Geospatial Consortium (Testbed-20 GeoDataCube API, Open Science Persistent Demonstrator).

This contribution will show our experiences in providing user-friendly computing and ESS services in an agile approach in the above-mentioned projects and discuss the challenges, which begin in finding a common language, in standardizing, combining or harmonizing existing solutions.