Application of multi modal deep learning framework for predicting the distribution of soil organic carbon

Accurately predicting the distribution of soil organic carbon (SOC) is essential for sustainable land management and climate change mitigation. However, due to the significant spatial variability of SOC and the complex interactions among soil factors, precise prediction remains a challenging task. With advancements in remote sensing technologies and increased data availability, various types of data have been utilized for SOC prediction. Nevertheless, traditional machine learning models often rely on single-modal data, which limits their ability to fully capture the complexity of SOC dynamics. Recent developments in deep learning have shown promise in improving environmental modeling by integrating multiple data sources. However, the effective integration of multi-modal data for SOC distribution prediction has not been fully explored. In this study, we proposed a multi-modal convolutional neural networks (MM-CNN) model that integrates satellite imagery and topographic variables derived from DEM to improve SOC prediction in the Walnut Creek watershed (WCW). Spatial features were extracted using CNN from an optical RGB band image captured by Google Earth on June 5, 2011, while 16 terrain variables derived from DEM were processed using artificial neural network (ANN) and concatenated with CNN features. The target variables include SOC density, Cesium-137 (¹³⁷Cs) inventory, and soil redistribution (SR) rate, which were obtained from 100 soil samples collected in WCW. To evaluate the performance of MM-CNN, we compared it with single-modal models, including CNN, ANN, and XGBoost, using the coefficient of determination (R²) and root mean squared error (RMSE) as performance metrics. Considering the spatial variability of SOC distribution, various image patch sizes centered on soil sampling points were used for both MM-CNN and CNN. The results of this study would show comparisons between MM-CNN and various single modal models predictions to inform the potential benefits of integrating complementary information from satellite imagery and topographic variables. The findings from this study would provide valuable insights of a multi modal approach for practical applications in environmental and agricultural fields.