Gridding crowd-sourced weather data: is quality control required?

Context. For monitoring, analysing and forecasting the impact of weather and climate change on society, we see an increasing need for high-quality, high-resolution gridded weather and climate services. The official networks of the National Meteorological and Hydrological Services (NMHS) sample part of the land-use excluding, for example, urban areas, which motivated us to include third-party data which does sample urban areas.  We have provided the results in real-time as well as historical gridded data services.

Approach. By including a statistical, simplified model of the observation error in our gridding approach (i.e. multi-fidelity regression Kriging), we did not include any advanced quality control (QC) of the crowd-sourced data. This might sound surprising in the context of weather and climate data,but is consistent with approaches in other scientific disciplines (e.g. marine engineering, aerospace engineering).

Results. In this study, we investigate the quantitative effect of state-of-the-art quality control on the accuracy of gridded services, by analysing hourly temperature observations in The Netherlands for the year 2023. Our results indicate that quality control of crowd-sourced weather data can potentially increase the accuracy of straightforward – and commonly used – nearest-neighbour approximation, but generally deteriorates the accuracy of more advanced gridded services. This finding indicates that using strict QC procedures to turn crowd-sourced data into a dataset with similar fidelity as the NMHS-sourced data is not the way to go. Therefore, we do indeed continue to blend first-party data, crowd-sourced data and land-use data without applying any advanced quality control to the crowd-sourced data.

Ecosystem. We emphasize that crowd-sourced weather data only improves services when it is blended with a high-quality network of NMHS data and possibly with land-use data. In other words, the availability of crowd-sourced data does not remove the need for high-quality NMHS observation networks. Therefore, we do not intend to present crowd-sourced data as a stand-alone product; rather, it lives in an integrated ecosystem.