Can we use a Zero-Shot Learning Model for Shallow Landslide Detection? An Application of RemoteCLIP

Accurate and timely detection of landslides is crucial for effective response strategies. Traditionally, landslide detection has relied on supervised learning models that require extensive labeled datasets or on expert labour to delineate landslide bodies in remote sensing imagery. Our explorative study uses zero-shot learning models, specifically the RemoteCLIP framework, for detecting landslides without the use of task-specific training data. We focused on rainfall-triggered shallow landslides in Slovenia that occurred in August 2023 to test this model’s efficacy.

RemoteCLIP, a variant of the CLIP (Contrastive Language–Image Pre-training) model, leverages visual and textual semantic similarities to classify land surface features based on metadata and environmental cues extracted from remote sensing imagery.

The method involved processing post-event aerial orthoimages with 1 m spatial resolution to identify potential landslides. Raw heat-map outputs from RemoteCLIP were compared against recorded landslide incidents for evaluating detection capabilities. The results revealed that RemoteCLIP effectively identified several major landslide sites, though performance fluctuated with prompt input and terrain complexity.

Our findings highlight the dependence of text-image foundation model performance predominantly on prompt design. Furthermore, they suggest opportunities for model refinement through inclusion of multimodal data inputs, like topographic information, as well as constraining the model output to physically plausible domains.

While being a first step, this research indicates the potential for zero-shot learning to transform landslide detection tasks by reducing dependence on large curated datasets, accelerating deployment in emergency situations, and enhancing responsiveness in data-scarce regions. Through further development, such AI-driven methodologies could provide valuable additions to current geohazard monitoring systems. We propose future work to focus on the integration of multimodal data beyond optical remote-sensing imagery and on the development of frameworks combining zero-shot models with more domain-specific classifiers and physical constraints.