RRR: Reliability, Replicability, Reproducibility for Climate Models

It is difficult to reliably build climate models, reproduce results and so replicate scientific findings. Modern software engineering coupled with the right tools can make this easier. 

Some sources of complexity that make this a difficult problem:  

• Climate models are an imperfect translation of extremely complex scientific understanding into computer code. Imperfect because many assumptions are made to make the problems tractable.  
• Climate models are typically a number of separate models of different realms of the earth system, which run independently while exchanging information at their boundaries.   
• Building multiple completely separate models and their many dependencies, all with varying standards of software engineering and architecture. 
• Computational complexity requires high performance computing (HPC) centres, which contain exotic hardware utilising specially tuned software.  

ACCESS-NRI uses spack, a build-from-source package manager that targets HPC, and which gives full build provenance and guaranteed build reproducibility. This makes building climate models easier and reliable. Continuous integration testing of build correctness and reproducibility, model replicability, and scientific reproducibility eliminates a source of complexity and uncertainty. The model is guaranteed to produce the same results from the same code, or modified code, when those changes should not alter answers.  

Scientists can be confident that any variation in their climate model experiments is due to factors under their control, rather than changes in software dependencies, or the tools used to build the model.