A modular data management approach for environmental observation campaigns in multiple cities

Field observation networks are becoming denser, more diverse, and more mobile, while being required to provide real-time results. The ERC urbisphere program is coordinating multiple field campaigns simultaneously to collect datasets on urban atmospheric and environmental conditions and processes in cities of different sizes. The datasets are used for improving climate and weather models and services, including assessing the impact of cities on the atmosphere (e.g. aerosols, greenhouse gases) as well as the exposure of urban populations in the context of atmospheric extreme events (heat waves, heavy precipitation, air pollution). For model development and evaluation, we are using meshed networks of in-situ observations with ground-based and airborne (remote-)sensing platforms. This contribution describes the urbisphere data management infrastructure and processes required to handle a variety of data streams from primarily novel modular observation systems deployed in complex urban environments.

The modular observation systems consist of short-term deployed instrumentation and are separated into three thematic modules. Module A aims at characterizing urban form and function affecting urban climates. Module B quantifies the impact of urban emissions (heat, pollutants, greenhouse gases, etc.) on the urban boundary layer over and downwind of cities. Module C provides data on human exposure at street and indoor-level. The three modules are served with consistent data management, documentation and calibration. Systems deployed in Modules A, B and C include customized automatic weather stations, (Doppler) lidars and ceilometers, scintillometers, balloon radio sounding and spectral camera imaging. Systems are street-light-mounted, located on building roof-tops or indoors as well as on mobile platforms (vehicles, drones). Data ingestion processes are automated, delivering moderate data volumes in real-time to central data infrastructure through mobile phone and IOT connectivity. A meta data system helps keep track of the location and configuration of all deployed components and forms the backbone for conversion of instrument records into location-aware, conventions-aligned and quality-assured F.A.I.R. data products. Furthermore, the data management infrastructure provides services (APIs, Apps, IDEs, etc.) for data inspection and computations by scientists and students involved in the campaigns. Select datasets are integrated in near real-time into other global or local data systems such as, e.g., AERONET, the Phenocam Network, ICOS, or PANAME, for multiple uses.

Besides technical aspects and design considerations, we discuss how cooperation and attribution are safeguarded when data are being accessed for immediate academic and citizen data science.