The 'Mine-the-Gaps' geospatial web application for visualizing and evaluating regional environmental data estimates

We present the 'Mine-the-Gaps' geospatial web application and use it to present our recently published series of cleaned and imputed environmental datasets. The Mine-the-Gaps tool positions environment sensor and regional estimation data side-by-side on a map, allowing researchers to visually check sensor locations, regional coverage, and the likely accuracy of any regional approximation methods. Users can upload their own time-series geospatial data datasets, but as a proof-of-concept, we present an online version of this tool (http://minethegaps.manchester.ac.uk) pre-loaded with our recently published environmental dataset. This contains 5 years of cleaned and pre-processed UK sensor data, originally from DEFRA (air quality) and the Met Office (pollen and weather) monitoring stations alongside our basic region estimations.

Sensors can’t be installed everywhere, so many areas are sparsely covered, with some measurements (e.g. pollens and SO2) even more sparse than others. Mine-the-Gaps allows users to, for example, approximate the level of pollen for somebody reporting severe allergy symptoms in Manchester if the nearest sensor is in Chester. Two basic, distance-based estimation algorithms are pre-loaded but more importantly, we provide researchers with methods to visualize, evaluate and compare new regional estimation techniques. Estimated data can be loaded into the application via a CSV file, or new algorithms can be added to our region estimations Python package, used by Mine-the-Gaps to calculate estimations on the fly. Uploaded data is presented on a map and time-series charts can be displayed that compare sensor data with approximations for that location, had the sensor been missing.

If users have their own sensor and/or region datasets, these can be uploaded to Mine-the-Gaps, which can be cloned from our code repository. Just 2 lines of script are required to get it running locally and accessible from any browser. Time-series sensor data from any location can then be uploaded and estimations made for any regions.

We publish Mine-the-Gaps, our environmental data-sets, and associated tools with a focus on making these resources as accessible and adaptable as possible, thus allowing researchers to get on with the key job of evaluating the impact and variance of environmental data, rather than becoming bogged down in pre-processing. Importantly, the emphasis on a transparent data processing and visualization methodology enables researchers to determine the usefulness, or not, of any technique used for mapping single site measurements to represent a specific geographical region.

Potential use cases include the evaluation of current and new regional estimation methods for sensor data; evaluating the effects of variation in region shape and size when making estimations; helping to determine where a new sensor would be best positioned to fit with the existing networks; the visualization of irregularly spaced datasets; and visualizing sensor data over time.