Assessing high-performance GPU programming in a high-level language

We are developing differentiable multi-physics solvers for extreme-scale geophysical simulations on GPUs (∂GPU4GEO project). These solvers exploit the massive parallelism of graphics processing units to significantly accelerate computations in geodynamic and related modelling applications.

As the hardware landscape evolves rapidly and GPUs are continuously enhanced with new capabilities, systematic performance benchmarking is required to better understand performance opportunities and limitations, particularly for complex workloads such as coupled Stokes and multi-physics solvers.

The Julia programming language, together with the JuliaGPU GitHub organisation, provides a unified framework for GPU programming in a high-level language. Owing to its expressive syntax and flexible compiler infrastructure, Julia enables productive development of GPU-accelerated codes without compromising performance.

We present performance benchmark results for GPUs from major vendors and assess their relative performance. We investigate benchmarks relevant for Geodynamics and ice flow codes, and consider forward and reverse passes on kernels as this is relevant for automatic gradient generation using automatic differentiation (AD). We further discuss performance per GPU price, which is relevant when evaluating the acquisition of local research computing infrastructure.