Weather prediction centers are always looking for the best computational performance for their numerical weather prediction (NWP) model, given their financial budget. Over the last decades, most centers relied on computer systems with scalar x86 architectures. This, however, might not be the best choice for the mid-term future, as the development of CPUs with ever-increasing performance and memory bandwidth is slowing down.
Nowadays, hardware manufacturers advertise massively multiprocessing GPUs as one future pathway. Unfortunately, GPUs have their own programming paradigms, as they are still requiring a CPU as their driver and bring their own memory. This necessitates significant adaptions of existing codes. Porting a large and continuously developed community code, such as ICON, to emerging hardware architectures poses its own special challenges.
Many parts of the ICON framework have been made ready for GPU systems in a multi-institute effort over the past years. MeteoSwiss plans to use GPU ICON operationally for limited area forecasts in 2023. Current development activities also make ICON-GPU ready to support the enhanced feature set used operationally by the DWD (such as grid nesting and parametrizations for global simulations). It was decided to port ICON by introducing OpenACC compiler directives to the FORTRAN code. This iterative development model makes it possible to merge ported code back directly into the main code repository and to stay up-to-date with other developments like in model physics. A tolerance based testing suite was deployed to make sure that ported features remain functional also when non-GPU-related changes are introduced to already ported code sections.
We present the general porting strategy and the current state of the port. We discuss specific optimizations and the lessons learned while porting an actively developed code. Finally, we present the performance on current GPU and CPU machines and compare them to the currently operational setup on the DWD vector supercomputer.
How to cite: Jacob, M., Alexeev, D., Dietlicher, R., Cherkas, V., Germann, E., Gessler, F., Hupp, D., Jocksch, A., Lapillonne, X., Müller, C., Osuna, C., Reinert, D., Sawyer, W., Schättler, U., and Zängl, G.: ICON NWP on GPUs, EMS Annual Meeting 2022, Bonn, Germany, 5–9 Sep 2022, EMS2022-138, https://doi.org/10.5194/ems2022-138, 2022.
