The Sentinels EOPF Toolkit: Driving Community Adoption of the Zarr data format for Copernicus Sentinel Data

The Standard Archive Format for Europe (SAFE) specification has been the established approach to publishing Copernicus Sentinel data products for over a decade. While SAFE has pushed the ecosystem forward through new ways to search and access the data, it is not ideal for processing large volumes of data using cloud computing. Over the last few years, data standards like STAC and cloud-native data formats like Zarr and COGs have revolutionised how scientific communities work with large-scale geospatial data and are becoming a key component of new data spaces, especially for cloud-based systems.

The ESA Copernicus Earth Observation Processor Framework (EOPF) will be providing access to “live” sample data from the Copernicus Sentinel missions -1, -2 and -3 in the new Zarr data format. This set of reprocessed data allows users to try out accessing and processing data in the new format and experiencing the benefits thereof with their own workflows.

This presentation introduces a community-driven toolkit that facilitates the adoption of the Zarr data format for Copernicus Sentinel data. The creation of this toolkit was driven by several motivating questions: 

• What common challenges do users face and how can we help them overcome them? 
• What resources would make it easier for Sentinel data users to use the new Zarr data format? 
• How can we foster a community of users who will actively contribute to the creation of this toolkit and support each other?

The Sentinels EOPF Toolkit team, comprising Development Seed, SparkGeo and thriveGEO, together with a group of champion users (early-adopters), are creating a set of Jupyter Notebooks and plug-ins that showcase the use of Zarr format Sentinel data for applications across multiple domains. In addition, community engagement activities such as a notebook competition and social media outreach will bring Sentinel users together and spark interaction with the new data format in a creative yet supportive environment. Such community and user adoption efforts are necessary in order to overcome adoption and uptake barriers and to build up trust and excitement to try out new technologies and new developments around data spaces.

In addition to introducing the Sentinels EOPF Toolkit, this presentation will also highlight lessons learned from working closely with users on barriers they face in adopting the new Zarr format and how to address them.