Global climate simulations at 2.8 km on GPU with the ICON model

The ICON modelling framework is a unified numerical weather and climate model used for applications ranging from operational numerical weather prediction to low and high resolution climate projection. In view of further pushing the frontier of possible applications and to make use of the latest evolution in hardware technologies, parts of the model were recently adapted to run on heterogeneous GPU system. This initial GPU port focus on components required for high-resolution climate application, and allow considering multi-years simulations at 2.8 km on the Piz Daint heterogeneous supercomputer. These simulations are planned as part of the QUIBICC project “The Quasi-Biennial Oscillation (QBO) in a changing climate”, which propose to investigate effects of climate change on the dynamics of the QBO.

Because of the low compute intensity of atmospheric model the cost of data transfer between CPU and GPU at every step of the time integration would be prohibitive if only some components would be ported to the accelerator. We therefore present a full port strategy where all components required for the simulations are running on the GPU. For the dynamics, most of the physical parameterizations and infrastructure code the OpenACC compiler directives are used. For the soil parameterization, a Fortran based domain specific language (DSL) the CLAW-DSL has been considered. We discuss the challenges associated to port a large community code, about 1 million lines of code, as well as to run simulations on large-scale system at 2.8 km horizontal resolution in terms of run time and I/O constraints. We show performance comparison of the full model on CPU and GPU, achieving a speed up factor of approximately 5x, as well as scaling results on up to 2000 GPU nodes. Finally we discuss challenges and planned development regarding performance portability and high level DSL which will be used with the ICON model in the near future.