Towards a Foundation Model for Global Terrestrial 3D Above and Below Ground Carbon Stock Mapping (3D-ABC)

Understanding the global carbon budget with its carbon sources and sinks is scientifically important and economically relevant. In particular, vegetation and soils are major and highly dynamic carbon pools in the Earth System and a substantial part of the terrestrial carbon budget is influenced by land use changes, vegetation dynamics, and soil processes.

Recent advances in Foundation Models (FMs) are transforming AI, enabling remarkable generalization and zero-shot learning capabilities. Within the Helmholtz Foundation Model Initiative, we are developing the 3D-ABC FM, a tool targeting the accurate mapping of above- and below-ground carbon stocks in vegetation and soils at high spatial resolution. 3D-ABC aims to provide a seamless understanding of terrestrial carbon distribution by integrating multimodal remote sensing, climate, and elevation datasets, and addressing complex challenges such as multi-dimensionality and multi-resolution in FMs. Our unique 3D-ABC partnership brings together key capacities from the domains remote sensing, carbon monitoring, AI, and high-performance computing to take on such FM development.

The 3D-ABC FM integrates large-scale remote sensing data, including multispectral satellite imagery from the Harmonized Landsat-Sentinel-2 (HLS) dataset, TanDEM-X InSAR coherence data, and 3D lidar data from space (GEDI, ICESat 1&2), aircraft, and ground-based platforms. We also aim to incorporate ERA-5 Land climate reanalysis information, GLO-30 digital elevation data, as well as local lidar and field data on vegetation, soils, and carbon flux parameters. High-resolution forest models will be used to benchmark carbon fluxes.

To accommodate the diverse data modalities assembled for 3D-ABC and to address eight selected downstream tasks, the AI model employs an adaptive architecture, integrating a multi-modal input processor, an FM encoder, an adaptive fusion neck, and task-specific prediction heads. The multi-modal input processor handles data with varying spectral dimensions, automatically mapping inputs to a unified feature space. The FM encoder extracts generalized deep features from the normalized inputs, which are then integrated into universal feature representations through the adaptive fusion neck. This fusion enhances interactions across modalities. Finally, the universal features are decoded into various outputs tailored to the specific needs of downstream tasks. In the first FM training phase, a pretraining strategy leverages a masked autoencoder to train the multi-modal input processor, the encoder, and the fusion neck in an unsupervised manner, enabling the model to develop robust representation capabilities. In the second phase, by leveraging the principles of transfer learning, the pretrained model is fine-tuned using labeled data from various downstream tasks.

3D-ABC targets use of the JUWELS Booster and JUPITER high-performance computing (HPC) systems located at the Jülich Supercomputing Centre (JSC). The JUWELS Booster comprises 936 compute nodes, each equipped with four NVIDIA A100 GPUs. JUPITER, the first European exascale supercomputer, is currently being installed at JSC. Its Booster module will consist of ~6,000 compute nodes, each featuring four NVIDIA GH200 GPUs. To maximize efficient JUPITER utilization, 3D-ABC is leveraging the JUPITER Research and Early Access Program, which provides early access for code optimization and preparation to ensure FM applications are optimized and ready for deployment when the system becomes operational in 2025.