Enhancing environmental indicator trustworthiness: A framework for user-specific quality assessment of spatial input data using data-fitness-for-purpose principles

The reputation of data providers and the accuracy of geodata and data sources are critical factors for the trustworthiness of environmental metrics and indicators to support policy making. In this context, the possibility of user-specific assessment of the quality of input data for composite indicator calculations using data-fitness-for-use and data-fitness-for-purpose approaches has gained importance. Data-fitness-for-use criteria describe the general suitability of a data set for further use and thus refer to the intrinsic quality of the data such as completeness, accuracy, consistency, and timeliness. Data-fitness-for-purpose criteria are more context-dependent and emphasise the suitability of data for a specific application according to the suitability of the user's requirements.

This contribution first analyses spatiotemporally dynamic input datasets for the derivation of environmental indicators to determine whether they contain sufficient quality information to enrich the indicators with data quality information. We consider two crop-type classifications for the derivation of a biodiversity indicator and phenological and meteorological data for the derivation of an extreme weather indicator. The available information on the quality of the input data represents established key metrics for the production-oriented assessment of thematic accuracy. A prerequisite for their calculation are reference data that are accepted as true and which are often not available for geodata derivatives such as composite indicators. If reference data are not available, we show that such overall input data accuracy metrics are not sufficient to derive quality metrics for subsequent products, such as composite indicators, as they lack information on the spatial distribution of accuracy.

Secondly, the structure of an open framework is discussed that allows the extension of geospatial quality standards such as ISO 19157-1 (2023). There, production-orientated thematic accuracy sub-elements such as “classification correctness or “quantitative attribute accuracy” are already included. In contrast, the aspects of “usability” for geodata remain undefined due to their heterogeneous nature. As a possible key sub-element, we propose spatial uncertainty metrics as an additional data layer, often derived as a by-product of modelling, which can support usability assessments and the communication of local and spatially aggregated indicator uncertainties.

In conclusion, the approach presented focuses on data quality, usability and standardisation, which is closely related to the FAIR principles, and emphasises the importance of making geospatial data and environmental indicators more reusable. We believe that this approach can significantly increase the value and utility of research objects in the earth and environmental sciences and foster their reuse in the context of science policy frameworks. The packaging of research objects together with quality assessments in a lightweight container format such as RO-Crate and Annotated Research Context facilitates in addition the (semi-)autonomous processing of these data by machines and thus their AI Readiness. 

 

ISO 19157-1 (2023). Geographic information: Data quality. International Organization for Standardization. Geneva, Switzerland. https://www.iso.org/standard/78900.html