PID-Driven Global Access to Flagship km-scale Climate Simulation Data

Climate science enterprise both produces and depends on extremely large datasets in order to meet the needs of diverse scientific and downstream user communities, especially as climate models are increasingly run at kilometre-scale resolutions, resulting in rapidly growing data volumes which increase demands on data handling infrastructures. Individual flagship simulations are no longer used by a single research group, but are routinely reused by dozens or even hundreds of researchers globally. Consequently, data findability, accessibility and reuse must be straightforward, data provenance must be transparent, and the full heritage of simulation data should be preserved in a machine-actionable manner to ensure scientific rigour, explainability and reproducibility.

In this contribution, we present a conceptual infrastructure-level approach developed within the WarmWorld project based on leveraging the versatility of globally unique persistent identifiers (PIDs) to address these challenges. Specifically, we illustrate that by assigning handles to simulation datasets already at the point of production, simulation data stored locally at a HPC data center can become part of a globally interoperable data ecosystem. In our concept, handle profiles contain an URL at which the dataset can be opened. Further, machine-actionable metadata, such as the detailed provenance information describing the employed model configuration or a data reuse license and citation, would be available from the handle landing page. Thus, the motivation behind the approach we follow here is akin to that of the FDO specifications.

Finalized simulation datasets would be exposed through globally accessible SpatioTemporal Asset Catalogs (STAC), where PIDs serve as the authoritative entry point for discovery and access. Data access would be handled by system libraries that resolve storage locations across heterogeneous storage tiers. Crucially, data access shall be designed to be globally open without the need for credentials, reflecting a strong demand from the climate research community, as clearly demonstrated during the WCRP kilometre-scale hackathon (May 2025).

Systematic assignment and pragmatic leveraging of handles assigned to locally stored datasets can thus enable scalable and interoperable access to flagship climate datasets across infrastructures and communities, effectively integrating traditionally closed HPC data environments into the global data space and facilitating interoperability with other large-scale data holdings.