Measuring the social impact of weather warnings with event-based surveys

At the Swiss Federal Office of Meteorology and Climatology MeteoSwiss, the main goal of our warning system is to support the affected population and organizations in the best possible way to take the necessary measures to reduce the impact of extreme weather events. How can we measure whether our warnings achieve this goal and create societal value? A classical way of assessing the success of a warning system is to calculate the hit rate and false alarm rate. While this is very important, it is just a first step in measuring the value of the warning. Even the most perfect warning is not a guarantee for a reduction of the impact of the warned extreme event. A next step would be to assess how many individuals of the target group were reached and whether they were satisfied with the warnings and understood the information. However, the number of people reached and their average satisfaction alone still do not indicate how successful our warnings are. What we ultimately would like to know is whether our warnings lead to improved risk assessment, behavioural change and finally, to reduced costs and damages through extreme weather.

This contribution presents a concept for a standardized population survey that aims to provide quantifiable measures on the social impact of the warning. By drawing from methods of impact assessment in the non-profit sector, we differentiate between output, outcome and impact of our warnings and derive indicators for each of these levels. Data for these indicators will be collected through representative population surveys in the affected regions a few days after an extreme weather event occurred. During a pilot phase to be launched in fall of this year, we will assess the potential of these indicators and of different data collection methods (representative online survey vs. representative telephone interviews vs. online survey with users of our channels). Although data will not be available yet at the time of the conference, the presentation aims to present our approach and discuss opportunities as well as challenges and limits of measuring the value of our warnings by using event-based surveys.