Breaking down data sharing barriers and uplifting FAIR for climate data at scale

Developing climate models often requires the ability to access and share extremely large datasets (spanning tens to hundreds of terabytes) that are discoverable and optimised for high-performance computing (HPC) applications. This is a major challenge, as researchers frequently lack the storage resources and specialised support needed to ensure efficient data management and sharing practices across the full data life-cycle. The challenges of sharing data are evident even when dealing with curated datasets that are prepared for broad access, citation, and reuse. However, these challenges are amplified during the rapid and iterative stages of model development and prototyping. At this point, multiple versions of datasets must be shared and evaluated by a wide range of experts before the data is finalised and curated for public use. This iterative process requires robust infrastructure and coordination to avoid bottlenecks that can hinder progress.

To help overcome these barriers, Australia’s Climate Simulator (ACCESS-NRI) provides a dedicated merit allocation scheme for compute and storage resources. This includes 3PB of storage for datasets that are for use by community members to undertake scientific research, support model development and/or will be shared for reuse. Experience has shown, that if the usage of these storage resources is not managed then the data can quickly go from being an asset to a liability.  Therefore, to maximise the value of both the data and investment in storage, ACCESS-NRI has developed an approach for sharing datasets that is designed to support science and innovation while enhancing the current practices for making data more accessible and usable in accordance with the FAIR and CARE principles.

We will present the motivating use cases and show how this approach supports the model development cycle while making data and the science it underpins more transparent, open and accessible. This approach encourages data generators to transition their datasets from unmanaged, undocumented spaces into managed environments where curation and oversight are aligned with the data’s intended purpose and use. It acknowledges that supporting FAIR principles does not always require full curation to the standards of a long-term publication. Instead, it focuses on reducing barriers to data sharing by promoting active data management practices. These practices enhance discoverability, trust, and reliability, ensuring that shared data is fit for purpose without imposing unnecessary burdens.

ACCESS-NRI is a national research infrastructure (NRI) established to support the Australian Community Climate and Earth System Simulator, or ACCESS. The ACCESS suite of software and data outputs are essential tools used to simulate past and future climate, weather and Earth systems and to support research and decision making within Australia.

ACCESS-NRI's mission is to build an open collaborative infrastructure that will accelerate research in Earth system, climate and weather modelling as well as enable new research not currently possible. The facility brings together skills in software development, high-performance computing, data management and analysis to enhance the ACCESS modelling framework, making it easier to use and lowering the barrier for scientific innovation.