Between Heaven and Earth

As the Copernicus program matures, ever more gridded data becomes available to researchers to incorporate into their studies. This data is partially raw satellite data, but an increasing amount of derived products are becoming available. In addition, data from various terrestrial sources is being aggregated to gridded formats, enabling integration with products derived from satellite data.

The technologies available for provision, sharing and processing of gridded data have traditionally been developed by the EO community, with functionality tailored towards the requirements of satellite data. Where these technologies have been applied to the more terrestrial products, both derived from satellite data as well as that generated from terrestrial sources, gaps become apparent in the metadata provided. 

These gaps pertain to concepts not required for satellite data, as they are either not relevant, or have clear default values. For example in ISO 19123-1:2023, while one can define if the value being provided pertains to the center of the grid cell or one of the corners (Pixel-in-center, pixel-in-corner), it is not possible to indicate that the value pertains to the entire area of the cell, as required for land cover or population grids.

A further gap becomes apparent regarding the Observable Property that is being conveyed by the provided data. When dealing with satellite data, the only Observable Property being provided tends to be radiance, the only additional metadata to be provided details the individual frequency bands. When dealing with terrestrial products there are almost infinite lists of Observable Properties for which data is collected or generated, clean indication of what exactly the data represents is essential.

In some cases such information is provided through the use of relevant extensions, e.g. the STAC raster extension, that foresees a link to a semantic resource defining what the data actually represents. However, often, this information is not provided in a structured form. The user must extract this information from textual documentation to understand what the data actually represents. 

Proper provision of Observable Property concepts with gridded data would greatly enhance both data discoverability and reuse, as essential concepts describing the data are cleanly exposed, not requiring the user to guess from titles or poorly defined keywords. Proper integration of Observable Property concepts in core metadata structures would greatly increase the FAIRness of provided data.

Further issues encountered in sharing gridded data from diverse sources have to do with the currently available standardized web services and APIs. OGC WCS has been shown to have integral errors providing data over time, while work on OGC API - Coverage has yet to be completed. The openEO API is an interesting alternative, but as this is a processing API, deployment of this API purely for data accessibility entails a great deal of unnecessary overhead.

In conclusion, in order to reap the potential that can be gained from the diverse gridded data products emerging from both terrestrial and satellite sources, there are still a number of issues to be resolved, both in their description and accessibility.

This work was enabled by the FAIRiCUBE EU Horizon Project.