Verification of impact-based operational weather warnings at ZAMG using real-time fire brigade and eye-witness data

Development, verification and feedback of impact-based weather warnings require novel data and methods. Unlike meteorological data, impact information is often qualitative and subjective, and therefore needs some sort of quantification and objectivation. It is also inherently incomplete: an absence of reporting does not automatically imply an absence of impacts.
The reconciliation of impact information with conventional meteorological data demands a paradigm change. We designed and implemented a verification scheme around a backbone of weather-related fire brigade operations and eye-witness reports at ZAMG, the national meteorological service of Austria. Meteorological stations, radar and derived gridded data are conceptualized as a backstop to mitigate impact voids (possibly arising from a lack of vulnerability, exposure or simply a lack of reporting), but are not the primary basis anymore.
Operation data from fire brigade units across Austria are stored at civil protection authorities at federal state level and copied to ZAMG servers in real-time. Their crucial information is condensed into a few components: time, place, a keyword (from a predefined list of operations) and an optional free text field. This compact information is cross-checked with meteorological data to single out weather-related operations, which are then assigned to event types (rain, wind, snow, ice, or thunderstorm) and categorized into three different intensity levels („remarkable”, „severe” and „extreme”) according to an elaborated criteria catalogue. This quality management and refinement is performed in a three-stage procedure to utilize the dataset for different time scales and applications:
 „First guess” based on automatic filtering: available in real-time and used for an immediate adjustment of active warnings, if necessary;
 „Educated guess” based on a semi-manual plausibility check: timely available (ideally within a day) and used for an evaluation of latest warnings (including possible implications for follow-up warnings);
 Final classification based on a thorough manual quality control: available some days to weeks later and used for objective verification.
Eye-witnesses can report weather events and their impacts in real-time via a reporting app implemented at ZAMG (wettermelden.at). Reports from different sources and trustworthiness are funneled into a standardized API. Observations from the general public are treated like a „first guess”, those from trained observers like an „educated guess”, and are merged with the refined fire brigade data at the corresponding stages.
The weather event types are synchronized with our warning parameters to allow an objective verification of impact-based warnings. We illustrate our measures to convert these point-wise impact data into spatial impact information, to circumvent artifacts due to varying population density and to include the “safety net” of conventional meteorological data. Yellow, orange and red warnings are thereby translated into probabilities for certain scenarios, which are meaningful and intuitive for the general public and for civil protection authorities.