Weed Segmentation in Fallow Farmlands Using the ANYSAT Foundation Model

Accurate mapping of weed infestations in fallow farmlands is critical for supporting sustainable weed management and reducing unnecessary chemical inputs. Previous work has demonstrated the effectiveness of convolutional encoder–decoder architectures, such as U-Net, for weed segmentation from satellite imagery; however, these approaches are typically constrained by sensor-specific training, limited cross-site generalisation, and sensitivity to variations in spectral and spatial resolution.  In this study, we investigate the application of the ANYSAT foundation model [1] for sensor-agnostic weed segmentation across heterogeneous fallow agricultural farms across Australia. Building on top of an established U-Net-based workflow, we evaluate whether a foundation model pretrained on diverse Earth observation data can improve robustness and transferability across multiple satellite sensors without explicit sensor-dependent retraining. Multi-spectral satellite imagery from different platforms is used to fine-tune ANYSAT for semantic segmentation of weed presence in fallow paddocks, with human-curated and U-Net-refined weed masks serving as supervisory labels.  We design a systematic evaluation strategy based on leave-one-farm and leave-one-region validation to test model robustness under spatial and spectral variability. Rather than focusing on achieved performance, this work emphasises assessing feasibility, identifying the strengths and limitations of foundation-model-based segmentation for this task, and outlining key considerations for operational deployment in data-sparse agricultural settings. By framing weed detection as a sensor-agnostic problem, this study provides a structured pathway for testing foundation models in agroecosystem monitoring. It contributes to understanding how emerging Earth observation foundation models can be adapted for practical agricultural applications. 

[1] Astruc, Guillaume, et al. "AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.