Multi-GPU accelerated parallel modelling for geophysical electromagnetic inversions

The forward modelling for curl-curl equations is the fundament for time-harmonic electromagnetic (EM) problems in geophysics. The simulations with the discretized partial differential equations (PDE) are computationally intensive and crucial to practical geophysical EM problems like Magnetotelluric/Controlled Source EM. However, most published algorithms for curl-curl PDEs are still CPU-based and cannot utilize the rapid development of modern large-scale multi-GPU parallel architectures. Based on previously proposed CPU-based divergence-free modelling algorithm, we develop a hybrid parallel paradigm to exploit the high-throughput of interconnected heterogenous parallel systems equipped with multiple GPUs. The large sparse linear system derived from the staggered-grid finite-difference approximation of curl-curl problem is decomposed into sub-domains and solved efficiently with a mixed-precision Krylov subspace GPU algorithm in parallel. To demonstrate how the practical inversion problems can be substantially accelerated, we test the new algorithm with both the synthetic and real-world 3D models, for forward/adjoint calculations. The test results show a promising ~3.5x improvement regarding the computation speed on a small NVIDIA® HGX based system, over a conventional CPU-base server cluster with 12 nodes. This may significantly reduce the computation time and carbon footage for large-scale frequency domain EM inversion problems and brings the possibility of near real-time EM imaging, as in engineering and environmental applications.