A User-oriented approach to verifying weather warnings

Verifying the quality of weather warnings is crucial to improving warnings, building trust, and enabling stakeholders to choose the right course of action in the face of a warning. In the context of the renewal of the warning system at Deutscher Wetterdienst (i.e., the RainBoW program: “Risk-based, Application-oriented and Individualizable Provision of Optimized Warning Information”) with the main goal to align weather warnings more strongly with the needs of their users, there is a need for verification approaches which are specifically user-oriented.

The recipients of weather warnings, such as emergency response teams, commercial actors, or the general public, however, have different preferences as to which aspects of weather warnings are the most important to them. This could be valuing the correct intensity of an event over its correct timing in a forecast, or the correct location over the correct duration. It could also mean that the trade-off between over- and under-forecasted weather events could vary between stakeholders as some might prioritize minimizing missed events (i.e., increasing the hit rate, POD) while others prefer not having too many false alarms (i.e., reducing the False Alarm Rate, FAR). Incorporating these user-specific requirements provides the potential to build on common verification methods and expanding them, making them more applicable and useful to individual stakeholders.

Additionally, communicating the results in a way that recipients understand and benefit from information on forecast quality is an important step in a user-oriented verification approach. This likely requires translating complex statistical information into real-world examples that are comprehensible for non-academic stakeholders as well.

Therefore, this work showcases the efforts made at Deutscher Wetterdienst to provide user-oriented weather warning verification by

• conducting small surveys to find out which aspects of weather warnings are more important to a certain user group,
• testing verification methods that take these preferences into account when evaluating the accuracy of a warning,
• developing and testing comprehensible and user-focused ways of communicating the quality of warnings to user-groups.