Automated Verification of Rain Warnings at MeteoSwiss

Historically, weather warnings at MeteoSwiss are verified manually by a team of forecasters. This method effectively leverages expert knowledge and has proven successful over the years. However, the granularity of the results and consistencyof this manual verification is limited and could only be increased at substantial costs. We have therefore developed an automatic verification framework that allows to evaluate warning quality at increased granularity both in space and time, i.e. at individual regions and for different lead times.

Our automatic verification algorithm first creates two sets of reference warnings to verify the forecaster issued warnings against. Both use a gridded precipitation data set from combined radar and rain gauges observations to assign a warning level to each one of Switzerland’s 159 warning regions (varying in size between 105 and 475 km²). The first set determines the optimal warning level and duration for each warning region. The second determines the optimal aggregated warning level, following forecasters’ established guidelines on minimal warning areas (1000 km²) and homogeneous durations across regions for warning events, analogously to how forecasters themselves aggregate weather forecast information when issuing warnings. Hence, while the first one leads to patchy warning maps that closely correspond to measurements, the second one results in smooth warning maps. These are further away from measurements, but more desirable from a communication perspective and account better for predictability limitations. In a second step, our algorithm compares the issued warnings with both sets of reference warnings: it classifies each hour as hit, miss, false alarm or correct rejection per region. Additionally, mechanisms for selecting the degree of tolerance regarding warning level and temporal accuracy are also implemented.

In this contribution, we present key findings from applying our algorithm to all rain warnings that were issued by MeteoSwiss between 2015 and 2025 and reflect on how we can use them to further improve our warnings. Additionally, we highlight how byproducts of our algorithm can be used to support the manual verification efforts. Finally, we will give an outlook on how we envision to include this algorithm in our operational verification suite.