ES2.2 | Communicating science and dealing with Uncertainties
Communicating science and dealing with Uncertainties
Conveners: Nadine Fleischhut, Vanessa Fundel, Gerald Fleming, Jelmer Jeuring, Bruno Joly, Ken Mylne
Orals Fri1
| Fri, 12 Sep, 09:00–10:30 (CEST)
 
Room E1+E2
Posters P-Thu
| Attendance Thu, 11 Sep, 16:00–17:15 (CEST) | Display Wed, 10 Sep, 08:00–Fri, 12 Sep, 13:00
 
Grand Hall, P51
Fri, 09:00
Thu, 16:00
Scientists communicate to non-peer audiences through numerous pathways including websites, blogs, public lectures, media interviews, and educational collaborations. A considerable amount of time and money is invested in this public engagement and these efforts are to a large extent responsible for the public perception of science. However, few incentives exist for researchers to optimize their communication practices to ensure effective outreach. This session encourages critical reflection on science communication practices and provides an opportunity for science communicators to share best practice and experiences with evaluation and research in this field.

DEALING with UNCERTAINTIES
This session will also include examples of how science can and should support decision-making. In this context a special section this year will be dedicated to the highly important issue of Dealing with Uncertainties:

Weather forecasts have matured substantially in providing reliable probabilistic predictions, with a useful quantification of forecast uncertainties. Including this information in the communication of forecasts and warnings, and integrating it into downstream models and decision-making processes has become increasingly common practice.

Including uncertainties not only implies the interpretation of ‘raw’ uncertainty information in ensemble forecasts, their post-processing, and visualization, but also the integration of a wide range of non-meteorological aspects such as vulnerability and exposure data to estimate risk and the social, psychological and economic aspects which affect human decision-making.

In this session, we aim to support a holistic perspective on issues that arise when making use of uncertainty information of weather forecasts in decision processes and applications.

Orals: Fri, 12 Sep, 09:00–10:30 | Room E1+E2

Chairpersons: Vanessa Fundel, Gerald Fleming, Jelmer Jeuring
09:00–09:15
|
EMS2025-427
|
Onsite presentation
Teresa Remes, Morten Køltzow, Andrew Singleton, Lene Østvand, Mette Sundvor Skjerdal, Bjørn Gilje Lillegraven, and Gunnar Noer

Weather forecasting is based on constantly evolving numerical weather prediction (NWP) models in different resolutions and time scales, and more recently also data-driven models based on machine learning. Successful application of these models depends not only on their objective performance skill but also on how forecasters understand, trust and communicate the model outputs and their uncertainties.

This contribution presents insights from a series of verification workshops involving operational duty forecasters (i.e., meteorologists on shift responsible for issuing weather warnings) at the Norwegian Meteorological Institute. The aim was to build dialogue between model developers and forecasters in order to develop verification systems, improve models, and create evaluation procedures that benefit both model development and forecasting. Specifically, the primary goal of these initial workshops was to better understand how forecasters evaluate model quality, which parameters and products are seen as the most challenging, and what conditions and tools are necessary for building trust in model products, including new data-driven forecasts. During the workshops, participants were invited to share their perspectives through discussions in groups and written feedback individually on sticky notes. The responses were later grouped and summarised to identify common findings.

Forecasters emphasized the importance of personal experience on duty, peer interaction, and case-based verification in building knowledge about model quality. Based on this, they have developed relatively high confidence in what NWP models represent well and poorly. For example, fog is consistently perceived as poorly represented in the models, whereas surface pressure fields and wind over the ocean are considered to be better represented. In order to build trust in data-driven forecasts, forecasters requested thorough and reliable verification with comparisons to the skill of traditional NWP models, especially with a focus on extreme events. Furthermore, practical experience over time, good training and transparency about how the data-driven models are trained and function were identified as key requirements.  

The findings highlight the need for structured training, communication of model strengths and weaknesses through multiple channels, and the co-development of tools that support forecasters in interpreting and communicating uncertainty. This work also emphasizes the importance of involving forecasters early in the development and integration of new forecasting tools.

How to cite: Remes, T., Køltzow, M., Singleton, A., Østvand, L., Skjerdal, M. S., Lillegraven, B. G., and Noer, G.: Forecasting with confidence: Forecaster perspectives from verification workshops at the Norwegian Meteorological Institute, EMS Annual Meeting 2025, Ljubljana, Slovenia, 7–12 Sep 2025, EMS2025-427, https://doi.org/10.5194/ems2025-427, 2025.

09:15–09:30
|
EMS2025-539
|
Online presentation
Andrea Taylor, Barbara Summers, and Sarah Jenkins

Impact based forecasts and warnings (IBFW) combined information about the potential consequences (impacts) of weather events with the estimated probability of them occurring. In representing IBFW ordered verbal categories such as low, medium and high may be used to classify both probability and impact severity. However, there has to date been limited exploration of how these categories are interpreted in the context of IBFW. While work in the social and behavioural sciences suggests that there can be variability in interpretation of verbal representations of probability, such as those used in IPCC reports, statements about impact severity – where numeric estimates may not be as easily attached to descriptions – remain under-investigated. In regionally representative survey with n > 1500 members of the public in England, UK, we asked participants to indicate on sliders how they would interpret a series of descriptors of likelihood (0=will definitely not happen,100=will definitely happen) and impact severity (0=not serious at all, 100=very serious) related to severe weather. Examining mean, median and distribution of responses for each verbal likelihood and impact descriptor we find that while descriptors indicating a middle point (i.e. medium, moderate) tended to elicit judgements around the central point of the scale for both impact and likelihood statements, there was overall less consistency around phrasing intended to denote low and high categories. For impact statements, we observe overlap between ‘very low’ (mean=33, median=26) and ‘low’ (mean=34, median=28) as descriptors. Catastrophic and extreme were interpreted to denote the highest level of impact, though in both cases mean and median ratings of impact seriousness were below 75 with indication of wide variability in interpretation. For likelihood, we find that only the descriptor ‘high’ elicited an estimated likelihood above 60. Surprisingly phrases that may be intended to denote very high likelihoods (‘expected’) or certainty/near-certainty (‘observed’) received mean interpretations close to the centre of the scale. Taken together our findings suggest that while terms typically associated with mid-points may be understood as intended, gaps between communicator and recipient may occur when discussion events with very low and very high likelihood. We discuss potential approaches to addressing this, including providing numeric values with likelihood descriptors and avoiding terminology where the greatest gaps between intention and interpretation may occur.

How to cite: Taylor, A., Summers, B., and Jenkins, S.: Describing likelihood and impact in impact-based forecasts: What’s likely? What’s severe?, EMS Annual Meeting 2025, Ljubljana, Slovenia, 7–12 Sep 2025, EMS2025-539, https://doi.org/10.5194/ems2025-539, 2025.

Show EMS2025-539 recording (13min) recording
09:30–09:45
|
EMS2025-627
|
Onsite presentation
A systematic review of approaches to evaluating communication effectiveness across phases in disaster risk management in Europe and USA
(withdrawn)
Aaron McKinnon and Imke Hoppe
09:45–10:00
|
EMS2025-355
|
Onsite presentation
Falk Anger, Dinah Leschzyk, Andreas Lambert, Bodo Erhardt, and Kathrin Feige

Enhancing user communication is a central goal of the RainBoW (Risk-based, Application-oriented and INdividualizaBle Provision of Optimized Warning Information) programme, which encompasses the development of the new weather warning system for the German Meteorological Service (Deutscher Wetterdienst, DWD). A clear understanding of issued weather warnings is essential for enabling both professionals and the general public to take appropriate action in affected areas. To support this, future warnings by the DWD will increasingly focus on impact information. Crowdsourced data, which reflects how users perceive specific weather events, offers a rich and valuable resource for improving this communication.

In this study, we focus on the hazards of wind gusts, which are among the weather elements with the greatest impact on infrastructure and society. To assess the severity of such events, we propose an impact proxy for Germany, derived from crowdsourced data collected through the Warnwetter App — the official weather app of the DWD. The app allows users nationwide to report their weather interpretations, providing a broad representation of the population’s exposure to weather events. Specifically, for wind gusts, users can choose from five predefined severity levels, ranging from "weak wind" to "severe storm".

We combine these crowdsourced reports (total number of roughly 55k) with meteorological ensemble model forecast data (ICON-D2-EPS) and geographic information (such as elevation and land use). Using statistical methods, we evaluate the suitability of this data as an impact proxy for wind gusts. While we find a strong correlation between user reports and model output, the data does not readily allow for a clear classification to predict user perceptions based on forecasts. We discuss potential ways to use this approach to improve communication with the public.

How to cite: Anger, F., Leschzyk, D., Lambert, A., Erhardt, B., and Feige, K.: Using Crowdsourced Data to Improve User Communication in Wind Gusts Warnings, EMS Annual Meeting 2025, Ljubljana, Slovenia, 7–12 Sep 2025, EMS2025-355, https://doi.org/10.5194/ems2025-355, 2025.

Show EMS2025-355 recording (13min) recording
10:00–10:15
|
EMS2025-677
|
Onsite presentation
Fatima Pillosu, Tim Hewson, and Ervin Zsoter

Flash floods are a significant societal problem that ranks as a top priority hazard for the World Meteorological Organisation and the United Nations. Pinpointing where and when they will hit is extremely challenging beyond lead times of an hour or two, even when using state-of-the-art convection-resolving ensembles, due mainly to significant ensemble size limitations. There has been more success in highlighting areas at risk from flash floods by post-processing numerical model output, either from these limited area ensembles, or from global ensembles with parametrised convection, or by blending the two.
A benefit of using global ensembles is that they are much less constrained spatially and in terms of lead times. One successful post-processing approach applied here has been the ECMWF “ecPoint” system. This can deliver finite probabilities for very large, localised totals that ordinarily the raw ensemble system cannot, and should not, predict itself. These have verified very well but could be considered less actionable by users because the probabilities delivered, for a point in a given gridbox, in advance of extreme events, are often very small (e.g. 1-5%). This presentation will outline three developments related to the ecPoint approach that make it more amenable to users by 1) providing an estimate of likely maxima within a gridbox, that 2) tailor better to flash flood risk than purely to rainfall totals by cross referencing a new global point-rainfall climatology, and that 3) demonstrate clear ‘financial’ utility even if probabilities are small, via computations of potential economic value. Case studies will be used for illustration.

How to cite: Pillosu, F., Hewson, T., and Zsoter, E.: Making Low Probability forecasts of High Impact Hydrological Events more useful for Society  , EMS Annual Meeting 2025, Ljubljana, Slovenia, 7–12 Sep 2025, EMS2025-677, https://doi.org/10.5194/ems2025-677, 2025.

10:15–10:30
|
EMS2025-688
|
Online presentation
|
Stefan Wolff, Jan Bondy, Norbert Demuth, Vanessa Fundel, Julia Keller, Andreas Lambert, Dinah Leschzyk, and Manuel Voigt

In July 2021, dramatic flooding in western Germany and Belgium led to more than 200 deaths and massive destruction. Due to the increasing impact of climate change, such events are becoming more likely in the future. The German Weather Service (DWD) and the Rhineland-Palatinate Flood Forecasting Centre (HVZ) launched the joint co-design project in order to raise awareness in this regard and increase the ability to act. One of the project objectives is to analyze and improve risk and hazard communication with a focus on flooding at all possible levels. In particular, a clear understanding of how to deal with uncertainties and probabilities is a key element for quick and targeted decisions.

In summer 2024, we conducted a survey aimed at all stakeholders affected by flooding. The goal of this survey was to obtain detailed feedback on the understanding of uncertainties and to determine the interest and willingness to participate in workshops, courses, e-learnings and the (joint) development of serious games. The feedback on this topic was overwhelmingly positive. Numerous interested parties have declared their willingness to support these developments. In two large online meetings, we then brought the interested parties together and carried out an initial analysis of ideas for the development of a serious game focusing on dealing with meteorological and hydrological uncertainties. The target groups for this game should be civil protection, administrations, schools and the interested public. We are planning the entire development process in close dialogue with the user groups, allowing us to incorporate specific experiences from the various areas of work. In our presentation, we would like to present the current state of development of this game and discuss possible steps for further improving communication.

How to cite: Wolff, S., Bondy, J., Demuth, N., Fundel, V., Keller, J., Lambert, A., Leschzyk, D., and Voigt, M.: Improving flood communication at all levels with the help of a serious game, EMS Annual Meeting 2025, Ljubljana, Slovenia, 7–12 Sep 2025, EMS2025-688, https://doi.org/10.5194/ems2025-688, 2025.

Show EMS2025-688 recording (14min) recording

Posters: Thu, 11 Sep, 16:00–17:15 | Grand Hall

Display time: Wed, 10 Sep, 08:00–Fri, 12 Sep, 13:00
P51
|
EMS2025-363
Christoph Sauter, Kathrin Wapler, Kathrin Feige, Mara Gehlen-Zeller, and Cristina Primo

Verifying the quality of weather warnings is crucial to improving warnings, building trust, and enabling stakeholders to choose the right course of action in the face of a warning. In the context of the renewal of the warning system at Deutscher Wetterdienst (i.e., the RainBoW program: “Risk-based, Application-oriented and Individualizable Provision of Optimized Warning Information”) with the main goal to align weather warnings more strongly with the needs of their users, there is a need for verification approaches which are specifically user-oriented.

The recipients of weather warnings, such as emergency response teams, commercial actors, or the general public, however, have different preferences as to which aspects of weather warnings are the most important to them. This could be valuing the correct intensity of an event over its correct timing in a forecast, or the correct location over the correct duration. It could also mean that the trade-off between over- and under-forecasted weather events could vary between stakeholders as some might prioritize minimizing missed events (i.e., increasing the hit rate, POD) while others prefer not having too many false alarms (i.e., reducing the False Alarm Rate, FAR). Incorporating these user-specific requirements provides the potential to build on common verification methods and expanding them, making them more applicable and useful to individual stakeholders.

Additionally, communicating the results in a way that recipients understand and benefit from information on forecast quality is an important step in a user-oriented verification approach. This likely requires translating complex statistical information into real-world examples that are comprehensible for non-academic stakeholders as well.

Therefore, this work showcases the efforts made at Deutscher Wetterdienst to provide user-oriented weather warning verification by

  • conducting small surveys to find out which aspects of weather warnings are more important to a certain user group,
  • testing verification methods that take these preferences into account when evaluating the accuracy of a warning,
  • developing and testing comprehensible and user-focused ways of communicating the quality of warnings to user-groups.

How to cite: Sauter, C., Wapler, K., Feige, K., Gehlen-Zeller, M., and Primo, C.: A User-oriented approach to verifying weather warnings, EMS Annual Meeting 2025, Ljubljana, Slovenia, 7–12 Sep 2025, EMS2025-363, https://doi.org/10.5194/ems2025-363, 2025.