GHOST: A globally harmonised dataset of surface atmospheric composition measurements

A critical measure for our understanding of the complex non-linear processes which determine atmospheric composition is through the use of Chemical Transport Models and Earth System Models. In order to evaluate the veracity of these models, observations are required, however the availability and quality of these observations serves as a major impediment to this process. The most temporally consistent measurements have been made at the surface by established measurement networks, typically for the purpose of monitoring local exceedances of air quality limits. There are multiple networks which report this data, in disparate formats, requiring harmonisation to allow for synthesis.

On the occasion that evaluation efforts use data from multiple networks, there is typically little to no detail given about the methodology used for the data synthesis across the different networks, or regarding the quality assurance (QA) or station classifications employed to subset the data. Therefore, evaluation efforts across different research groups are often incomparable.

As a response to this common challenge, we established GHOST (Globally Harmonised Observational Surface Treatment). GHOST can be succinctly stated as an effort to standardise the data / metadata from the major public reporting networks which provide in situ atmospheric measurements at the surface. In total the dataset comprises of ~20 billion processed measurements, for ~200 components, across 32 networks, from 1970 to 2022. This represents the biggest collection of harmonised atmospheric composition surface measurements ever composed. 

Substantial efforts have been made towards standardising almost every facet of provided data / metadata from across the networks. On top of this, additional metadata was added by processing various commonly utilised globally gridded datasets (e.g. land use), as well as adding temporal classifications per measurement (e.g. weekday / weekend). As the dataset spans many decades, metadata is handled dynamically and allowed to vary through the record, important for instances when there are changes in measurement instrumentation or the measurement position.

Major efforts were made for the standardisation of the numerous metadata fields detailing measurement procedures, with all measurements linked to a dictionary of standard measurement methods and standard instruments. Great effort was also spent in the standardisation of station classifications, providing large flexibility for the subsetting of stations. Rather than dropping any measurements which are labelled as potentially erroneous by the measurement provider, standardised data flags are associated with each individual measurement. On top of this, GHOST own QA flags are also associated per measurement.

All data is now freely available to the community.