Spherical harmonic transforms up to degree 100,000 and beyond using distributed-memory systems

Thanks to existing algorithms for Legendre functions, it is fairly possible to perform spherical harmonic transforms up to degrees as high as a few tens of thousands. In fact, many algorithms are often accurate even well beyond this point. From the practical point of view, however, hardware-related challenges emerge when the truncation degree exceeds, say, 100,000. For instance, to conduct a degree-100,000 spherical harmonic transform, be it forward or backward, it is necessary to store in memory about 75 GBs of spherical harmonic coefficients and about 149 GBs of the signal to be analyzed/synthesized at the Gauss--Legendre grid (with Driscoll--Healy grids, the signal-related memory even doubles). Although systems with that amount of shared memory are not rare, the point becomes clear when further extending harmonic degree to, say, 150,000 (168 GB + 335 GB) or beyond. With such extensive transforms, one has to sooner or later move to systems with distributed memory.

This contribution discusses the implementation of spherical harmonic transforms on distributed-memory systems in CHarm, which is a C/Python library for high-degree spherical harmonic transforms. To this end, CHarm employs the Message Passing Interface (MPI), allowing to distribute both the coefficients and the signal among a number of independent shared-memory systems that can communicate together. These could be, for instance, nodes of high-performance computing clusters or a few ordinary PCs interconnected via some network protocol (e.g., SSH). Since the computation of a signal at any point requires all spherical harmonic coefficients and vice versa, the amount of data transferred is immense. We show how this challenge is tackled in CHarm. Furthermore, we discuss how CHarm combines the MPI parallelization (between shared-memory systems), the OpenMP parallelization (within shared-memory systems) and the SIMD parallelization (within a single CPU core), all at the same time. As a toy example, some spherical harmonic transforms up to degree 100,000 and beyond are shown.

In Physical Geodesy, high-degree spherical harmonic transforms are useful, for instance, to model planetary topographies. More specifically, spherical harmonic degree 100,000 offers 200-meter spatial resolution on the Earth's surface. Systems with distributed memory thus make it feasible for spherical harmonics to capture fine details of the Earth's and other planetary topographies. As another use case, it is known that topography truncated at some finite non-zero harmonic degree generates gravitational field possessing an infinite number of spherical harmonics. Therefore, to accurately describe the gravitation field of, say, a degree-10,800 Earth's topography, the truncation degree of the implied potential series should generally be extended from 10,800 to degree as high as possible, say, 108,000 or so.

CHarm is free software available from https://github.com/blazej-bucha/charm. The documentation with cookbook-style examples can be found at https://www.charmlib.org.

This study was funded by the EU NextGenerationEU through the Recovery and Resilience Plan for Slovakia under the project No. 09I03-03-V04-00273.