Recognizing any Land surface anomaly with multi-modal foundation model

Various Land surface anomalies have destroyed the stable and balanced state of human living, resulting in fatalities and serious destruction of property. Remote sensing technique has been proven useful in many studies with time-series and large-scale observation advantages. However, existing studies are limited in anomaly recognition of certain categories, lacking the important generalization ability to recognize rare or unseen anomalies. To tackle this problem, we have built a multi-modal land surface anomaly recognition foundation model, which connects the images and anomaly caption words in an open-world setting. A global scale multi-modal dataset is constructed to train the model, which refers to 1000 large-scale monitoring regions covering over 2000 km2 in total, with rich text caption collected from offical news report. After the self-supervised contrastive learning with image and text modalities, the foundation model can describe both the anomaly category and attributes directly given any monitoring image, without the need for further tuning. These open-world and tuning-free settings promote the ability of rapid anomaly monitoring.