Operational multi-model ensemble postprocessing: key challenges and lessons learned

MeteoSwiss has recently introduced multi-model ensemble postprocessing for the automatic production of its local forecasts. While this has allowed to streamline forecast production and resulted in improved forecasts compared with the legacy system, the inclusion of data-driven approaches in the forecasting process also poses unique challenges:

The statistical postprocessing produces probabilistic forecasts that are well calibrated. This often implies that forecast spread is increased compared with the underlying NWP forecasts requiring adaptation of the forecast visualizations and products. For example, producing deterministic weather symbols (pictograms) to summarize the weather evolution throughout the day is a challenge in particular during convective situations when spatio-temporal uncertainty in the forecast is very pronounced even a few hours ahead. A careful retuning of the decision tree to produce weather symbols was a key requirement for the successful introduction of postprocessed forecasts of precipitation and cloud cover.

Initially, the statistical postprocessing has been optimized for general-purpose rather than high-impact weather. In cases of weather warnings there may be notable inconsistencies between the automated forecast based on statistical postprocessing and the official warnings produced by the forecasters. Often, the statistical postprocessing underestimates the intensity of the event whereas the high-resolution NWP better reflects the observed situation. While we are currently further refining our postprocessing to adapt it to severe events, discrepancies between automated forecasts and manually tailored information will remain a communicative challenge. 

Finally, the operational  and organizational challenges of running data-driven approaches are not to be underestimated. Data-driven approaches have to be constantly maintained and monitored to avoid adverse impacts of erroneous observations and other sources of data drift. Furthermore, NWP development cycles and reforecasts for testing need to be co-designed to provide sufficient data for re-training of statistical approaches. Also, the added complexity poses a challenge when seeking to understand unexpected forecast behavior and respond to end-user feedback. While NWP characteristics are quite well known by all development and forecasting teams, knowledge on the specifics of the postprocessing is less well developed in the organization. 

While statistical postprocessing is a key component in automated forecasting, careful design of the system with a keen eye on operational constraints is necessary to manage the additional complexity. With the recent advent and proliferation of data-driven approaches in weather forecasting, we expect such considerations to become increasingly important.