Evaluation of ESTOFEX convective outlooks from 2007 to 2021. Part 1: forecasters and regional performance of lightning predictions

The European Storm Forecast Experiment (ESTOFEX) is a well known project of volunteer meteorologists making forecasts of thunderstorms and their severe weather threats for Europe for 20 years (since October 2002). Originally, the forecasts consisted of three severe weather risk lines as well as one thunderstorm line. The verification of the dichotomic lightning forecasting skill was presented at the 4^th ECSS in Trieste, 2007.

In 2009 ESTOFEX switched to the use of two thunderstorm probability lines, tentatively marked “15% probability” and “50% probability” of thunderstorms within 40 km from each location. A verification of these lightning probability forecasts over a 4-year period was presented at the 7^th ECSS in Helsinki in 2013 (doi: 10.13140/RG.2.1.1026.1845).

We update the verification of lightning probability areas to include 8 more years, covering 2009-2021 (incl.), and evaluate consistency of each forecaster by histograms and maps of thunderstorm occurrence (observed mean frequency for a location) for each category of forecast. Other aspects of this work will be presented in Part 2 (poster by Mateusz Taszarek). Gridded lightning data from ATDNET is used, as well as severe weather reports from the ESSL European Severe Weather Database. The ESTOFEX forecast category definitions are 0-15%, 15-50%, >50% forecasts for thunder, 0-5% (Level 1) and 5-15% (Level 2) for severe weather, and >15% (Level 3) for extremely severe weather, respectively.

As in the previous work, two key questions are answered:

(1) “Of all the times a location was included in a certain forecast category (e.g. 15-50% area), how often was the phenomenon observed?"

This returns a percentage for each location on the map. These locations can be plotted on the map or grouped into 3 overlapping histograms, one for each forecast category. The overlap and calibration can be judged for each forecaster and by geographic regions. The spatial criterion (e.g. phenomenon within 40 km from a point) can be varied.

(2) “If a spatial thunderstorm density was observed, which forecast probability area was it included in?"

For each category of observed spatial density around a point (or a Practically Perfect Hindcast, see Part 2), a map and histogram can be plotted of the corresponding mean forecast category frequency (or difference), to find areas where forecasters most frequently under- or overestimate activity, for example by season.