Improved understanding of warning evaluation: will you contribute to the survey?

From the perspective of National Meteorological and Hydrological Services verification is needed to assess the quality of severe weather warnings, to determine their strengths and weaknesses and consequently to lead to improvements. Apart from the challenges that arise when analysing rare meteorological events, a set of statistical methods are suitable for comparing weather forecasts or warnings with observations to assess the (physical) accuracy of forecasts. For impact-based warnings, the complexity is increased since it is not enough to compare the forecasted meteorological values against observed values. The societal consequences of the hazard (e.g. comparison with damage compensation statistics) should also be taken into account, as well as the preventive measures taken by various actors - based on the information in the warning - to reduce risk and damages.

 

What determines how valuable a warning is from a user’s point of view? Do users and providers have the same understanding of the value of a warning, and the thresholds for issuing warnings? How can the value of a warning be best communicated to users to support their decisions to mitigate risk or prepare for it as best as possible? Some potential factors to consider in an evaluation could be how accessible the warning is, timeliness, how understandable the information about the weather, the consequences and the risk is, and how useful the information is in their decision-making processes. The latter might be affected by e.g. how trustworthy they consider the (provider of the) warning to be, how skillful or accurate the information is, or how relevant the information is to their specific situation or context. Among users, e.g. emergency services, authorities in civil protection, media and communication, and the general public, there is a wide variety in the way warnings are used and understood and in requirements concerning severe weather warnings. 

 

A user-oriented verification and evaluation could help the NMHSs understand the capabilities and limits of warnings among the different user groups, and consequently improve their decision-making capabilities, such as safeguarding life and property.

 

Here, we will present results from a survey conducted at the second Weather and Society Conference (WMO) in February 2024, and at the seventh conference on weather warnings and communication (AMS) in June 2024. The respondents were experts from various fields, participating in the conference. They gave input on verification and evaluation activities in the field of weather warning and its communication to users, and opinions on how to best perform user-oriented evaluation of warnings. 

Further, to increase the impact of the study, we would also like to invite all conference participants at the EMS annual meeting in Barcelona to respond to the short survey (ten minutes), by accessing this URL: https://forms.gle/eBe6S8qeweL1LukV8.