Forecasting with confidence: Forecaster perspectives from verification workshops at the Norwegian Meteorological Institute

Weather forecasting is based on constantly evolving numerical weather prediction (NWP) models in different resolutions and time scales, and more recently also data-driven models based on machine learning. Successful application of these models depends not only on their objective performance skill but also on how forecasters understand, trust and communicate the model outputs and their uncertainties.

This contribution presents insights from a series of verification workshops involving operational duty forecasters (i.e., meteorologists on shift responsible for issuing weather warnings) at the Norwegian Meteorological Institute. The aim was to build dialogue between model developers and forecasters in order to develop verification systems, improve models, and create evaluation procedures that benefit both model development and forecasting. Specifically, the primary goal of these initial workshops was to better understand how forecasters evaluate model quality, which parameters and products are seen as the most challenging, and what conditions and tools are necessary for building trust in model products, including new data-driven forecasts. During the workshops, participants were invited to share their perspectives through discussions in groups and written feedback individually on sticky notes. The responses were later grouped and summarised to identify common findings.

Forecasters emphasized the importance of personal experience on duty, peer interaction, and case-based verification in building knowledge about model quality. Based on this, they have developed relatively high confidence in what NWP models represent well and poorly. For example, fog is consistently perceived as poorly represented in the models, whereas surface pressure fields and wind over the ocean are considered to be better represented. In order to build trust in data-driven forecasts, forecasters requested thorough and reliable verification with comparisons to the skill of traditional NWP models, especially with a focus on extreme events. Furthermore, practical experience over time, good training and transparency about how the data-driven models are trained and function were identified as key requirements.  

The findings highlight the need for structured training, communication of model strengths and weaknesses through multiple channels, and the co-development of tools that support forecasters in interpreting and communicating uncertainty. This work also emphasizes the importance of involving forecasters early in the development and integration of new forecasting tools.