Earth Observation embeddings at the test: A novel benchmark to evaluate (neural) compression for satellite imagery

With the rapidly growing production and utilization of Earth Observation (EO) data, the past decade sparked interest in the efficient compression of EO data into low-dimensional embeddings. In a parallel development, EO Foundation Models (FM), trained on large amounts of unlabeled data to be used in a wide range of applications, also utilize low-dimensional embeddings to distill representations of EO data [1, 2, 3]. In one aspect, EO FMs may serve as (lossy) neural compressors to improve data transfer and lower storage needs – effectively reducing the carbon footprint of EO data [4].

While the development in EO FMs rapidly advances, there is need for a novel benchmark scheme to evaluate the quality of (compressed) embeddings. The statement “foundational” or “general purpose representation” needs a test.

As part of the Horizon Europe project “Embed2Scale” [5], co-funded by the European Union (Horizon Europe contract No. 101131841), the Swiss State Secretariat for Education (SERI), and UK Research and Innovation (UKRI), we present a novel approach to benchmark learnt compression of multimodal Copernicus Sentinel data for various relevant application domains. In the form of a competition, contestants provide embeddings that are evaluated on a diverse set of problems based on real-life use cases relevant for the research community, governments, and corporate businesses. The problems are hidden from the contestants to evaluate the applicability of the embeddings to unknown problems. The benchmark statistically evaluates the performance of downstream tasks through fine-tuning of neural networks that fit into commodity hardware. We underline a practically relevant scenario where end users rarely have access to costly and energy-intensive acceleration hardware. The overall performance, i.e. the evaluation across all the benchmark’s problems, is crucial and ensures a diverse and fair evaluation of the embeddings. After the competition, the datasets in the benchmark are published and made available to the community.

[1] X. Sun et al., “RingMo: A remote sensing foundation model with masked image modeling,” IEEE Transactions on Geoscience and Remote Sensing, 2022.

[2] D. Wang et al., “Advancing plain vision transformer toward remote sensing foundation model,” IEEE Transactions on Geoscience and Remote Sensing, 2022.

[3] C. Bodnar et al., “Aurora: A foundation model of the atmosphere,” Tech. Rep., 2024.

[4] R. Wilkinson, M.M. Mleczko, R.J.W. Brewin, K.J. Gaston, M. Mueller, J.D. Shutler, X. Yan, K. Anderson, Environmental impacts of earth observation data in the constellation and cloud computing era,Science of The Total Environment, Volume 909,2024,168584,ISSN 0048-9697, https://doi.org/10.1016/j.scitotenv.2023.168584

[5] https://embed2scale.eu/