Identifying Heterogeneous Landslides using Multi-modal Deep Learning

Automated detection of landslides is an important part of geohazard prevention. In dense vegetation covered area, identifying landslides is a challenging problem. Various types of landslide monitoring technologies have generated heterogeneous data, such as optical imagery, SAR imagery, and LiDAR point clouds. Different types of landslide monitoring methods have their advantages and drawbacks. An ideal landslide detection model should utilize their advantages. However, the complementary information of multi-source landslide monitoring data has not been fully understood. To deal with this problem, we study how to use multi-source data for developing better landslide detection models. First, a multi-modal deep learning model is introduced for landslide detection using multi-source landslide monitoring data. Second, representation learning networks are proposed for extracting landslide detection features from optical imagery and LiDAR-derived data. In addition, an attention-based data fusion network is proposed for merging the feature maps of different data sources. Finally, to improve the explainability of the proposed neural network, a new loss function with domain knowledge constrains is proposed. The proposed multi-modal deep learning method is compared with the existing machine learning-based landslide detection methods. Experimental results demonstrated that the proposed method outperformed the state-of-the-art landslide detection methods, and is able to simultaneously identify earthquake-triggered new landslides and forest-covered ancient landslides. The reason is that optical imagery is appropriate for identifying new landslides, while LiDAR-derived data is able to remove forest cover and suitable for identifying ancient landslides. It can be seen that the complementary information of multi-source data is helpful for improving the performance of landslide detection.