XAI for small-data problems in remote sensing: monitoring Atlantic forests with UAVs

Despite the increased availability of UAV / drone imagery in Low- to upper  Middle-Income countries and the demonstrated potential of deep learning to support the interpretation of these images for sustainable development purposes, practical operations in these countries are constrained by the need for sufficient labeled data-sets which are often difficult to obtain (especially for tropical forest). This makes it difficult to train suitable networks and assess whether the model is performing well. One such example is the use of drones to monitor the Atlantic Forest in Sao Paulo, Brazil. Here, members of the Sao Paulo Municipal Green and Environment Secretariat (Secretaria do Verde e do Meio Ambiente - SVMA) are starting to use drones to  identify some native and invasive species in their forests. Deep learning will quickly speed up this process, but there is little training data available. This refers to the so called ‘small-data problem’ commonly found in DL for remote sensing applications [1]. A workflow was designed to support this application through a novel zero-shot learning technique and explainable AI methods. A pre-trained tree-crown detection model ‘DeepForest’ [2] is used to identify individual tree crowns in the UAV imagery. The detected tree-crowns are further classified using a Siamese network architecture using zero-shot learning – the model is trained on relevant data-sets but not exposed to species found in the test data-set. A Siamese network architecture is motivated by the need for explainability in DL models – the results will be used for making administrative decision for forest management. A more intricate DL model (such as image segmentation) could be more accurate but at the cost of transparency/explainability. In particular, we apply a variation of the ‘What I Know’ (WIK) explainability method [3] which provides examples from the training set along with the test sample increasing transparency and understanding of the model results.

[1] Safonova, Anastasiia, et al. "Ten deep learning techniques to address small data problems with remote sensing." International Journal of Applied Earth Observation and Geoinformation 125 (2023): 103569.

[2] Weinstein, Ben G., et al. "DeepForest: A Python package for RGB deep learning tree crown delineation." Methods in Ecology and Evolution 11.12 (2020): 1743-1751.

[3] Ishikawa, Shin-nosuke, et al. "Example-based explainable AI and its application for remote sensing image classification." International Journal of Applied Earth Observation and Geoinformation 118 (2023): 103215.