Crop Type Mapping Using Self-supervised Transformer with Energy-based Graph Optimization in Data-Poor Regions

Updated crop type information is essential for agricultural monitoring and irrigation management applications. However, despite their importance, crop-type datasets, especially recent in-season data are still unavailable in most of developing countries where food security is a major concern. The lack of information on crop types is due to the limitations of traditional field surveys, which are generally infeasible, expensive, and inconsistent over larger spatial scales. Although remote sensing and machine learning approaches can provide cost-effective solutions for automatic crop-type mapping, huge amounts of high-quality training data are required in order to develop such applications. Therefore, semi-supervised or unsupervised learning methods become a potential solution to overcome the issue of lacking labeled training data. 

In this work, we explored semi-supervised and self-supervised techniques to map crop types in data-poor regions such as Saudi Arabia. The Self-supervised Transformer with Energy-based Graph Optimization (STEGO) method is a transformer-based segmentation technique that has the capability of both discovering and segmenting objects without the need for labeled data or human intervention. We evaluated the capability of STEGO to classify crop fields using Sentinel2 images. These images consisted of the derived maximum normalized difference vegetation index (NDVI), the standard deviation of NDVI, and the green chlorophyll vegetation index (GCVI).  The STEGO approach uses a novel contrastive loss that helps to distill pre-trained unsupervised visual features into semantic clusters, which is reported to outperform other unsupervised clustering methods. We set k=5, where k is the number of classes that reflects 4 crop types and a background class. 

Preliminary results of STEGO applied to small agricultural regions in Aljouf which is located in the north of Saudi Arabia captured the difference between crops when analyzed visually. Furthermore, assigning a crop-type label to each cluster class can be a challenging task. As for now, a brute force approach is followed to find the best assignment, and that is the assignment that provides the best results. As well as referring to previous knowledge of major crops grown in the region of interest, for this region, the major crops were wheat, olives, and tomato, in addition to “other” and “background” classes, which make up the 5 classes. Further work includes quantifying the accuracy of the clustering performance using the mean intersection over union metric (mIoU) and examining the effects of regional and national upscaling on the performance.