Making FAIRness Visible: Practical FAIR Assessment for Earth System Science Data

The FAIR-by-design approach pursued by most repositories and data services today requires significant and sustained effort in the curation and quality assurance of both data and metadata. Beyond providing research data that complies with the FAIR principles, it is essential that the level of FAIRness is transparently apparent to users from the metadata prior to data access and download. FAIRness indicators benefit both data providers and reusers by rewarding high-quality curation and supporting informed data selection in  complex, data-intensive Earth System Science (ESS) workflows.

In practice, making FAIRness levels visible requires repository data managers to perform  FAIR evaluation, either through manual assessment or by using established FAIR assessment tools. At the World Data Center for Climate (WDCC) the fully automated F-UJI tool is applied in operational practice to assess and expose FAIRness levels across large collections of climate data.

F-UJI is a web based service that programmatically assess FAIRness of research data objects at the dataset level based on the FAIRsFAIR Data Object Assessment Metrics. Its   automated and machine-aided analytics are well suited for the large amounts of datasets archived in WDCC and reflect established repository practices such as the assignment of DataCite DOIs and the provision of rich, standardised metadata. At the same time, automated assessment relies on clearly machine-assessable criteria, and thus can not fully capture FAIR aspects that require human interpretation, such as reuse relevance or domain-specific semantics. In addition, FAIRness results depend on the machine-detectability of persistent identifiers resolving directly to datasets, which are not always available at higher levels of data collection hierarchies.

Based on our operational experience, we compare F-UJI results with other FAIR assessment approaches, building on findings from a previous comparative study evaluating FAIR assessment methods for WDCC datasets (Peters-von Gehlen et al., 2022). This comparison shows that automated, manual, and hybrid FAIR evaluation approaches each have distinct strengths: automated methods focus on standardised, machine-actionable criteria, while manual assessments capture contextual aspects relevant for data reuse; hybrid approaches combine these advantages and mitigate the limitations of purely automated or manual methods.

This poster shares practical experiences from conducting operational FAIRness assessment at a climate data repository and discusses benefits, limitations, and best practices of automated and hybrid FAIR evaluation approaches in Earth System Science.