OSA1.5 | Probabilistic and ensemble forecasting from short to seasonal time scales
Probabilistic and ensemble forecasting from short to seasonal time scales
Convener: Andrea Montani | Co-conveners: Jan Barkmeijer, Fernando Prates
| Mon, 04 Sep, 09:00–13:00 (CEST)|Lecture room B1.04
| Attendance Tue, 05 Sep, 16:00–17:15 (CEST) | Display Mon, 04 Sep, 09:00–Wed, 06 Sep, 09:00|Poster area 'Day room'
Orals |
Mon, 09:00
Tue, 16:00
The session will focus on the most recent developments in the field of ensemble techniques, ranging from its close connections with data-assimilation and nowcasting at short and medium ranges to their capacity to produce and deliver skillful and reliable forecasts of high-impact extreme events at sub-seasonal to seasonal (S2S) timescales.
As such it may provide a platform for exchanging ideas on how to create and use an ensemble system, techniques varying according to the forecast lead time. In particular, the forecaster perspective and the use of ensembles in predicting hazardous weather will be of interest.

The conveners invite papers on various issues associated with Ensemble Forecasting for weather prediction, such as:
• representation of initial uncertainties in Global and Limited-Area Ensemble Prediction Systems, including interlinks between data-assimilation and probabilistic forecasting;
• representation of model or boundary uncertainties in Global and Limited-Area Ensemble Prediction Systems;
• results from experiments including THORPEX Regional Campaigns, HyMeX, FROST-2014, etc.;
• results from recent studies using TIGGE and TIGGE-LAM databases;
• use, verification and calibration methods of Ensemble Prediction Systems;
• applications of probabilistic forecasts in the sectors of energy, health, transport, agriculture, insurance and finance;
• challenges tackled by the S2S WWRP/THORPEX-WCRP joint project (http://s2sprediction.net), including discussion on S2S sources of predictability, forecasts and socioeconomic applications of high-impact climate services.

Participants are especially encouraged to present contributions and discuss strategies to bridge gaps between stakeholders and actionable S2S tailored products.

In this session, the Award Lecture for the EMS Technology Achievement Award 2023 will be given:

11:00 - 11:30: Yr service development: A new, seamless, 3-week forecast
by Anders Sivle, Norwegian Meteorological Institute

Orals: Mon, 4 Sep | Lecture room B1.04

Chairperson: Andrea Montani
Onsite presentation
Ulrich Blahak and the Team SINFONY

The development of DWD's new Seamless INtegrated FOrecastiNg sYstem (SINFONY) has matured and a number of components have been run  continuously in near-realtime on a daily basis during the last two convective seasons. This presentation will give a short overview on the system components itself and their current status, and will report on the results of this years convective season.

There are different "optimal" forecast methods for different forecast lead times and different weather phenomena.
Focusing on precipitation and convective events up to some hours ahead, we developed

  • a) radar Nowcasting ensembles for areal precipitation, reflectivity and convective cell objects,
  • b) a regional ICON-ensemble model with extensive data assimilation of high-resolution remote sensing data (3D radar volume scans of radial winds and reflectivity, cell objects, Meteosat VIS channels and lightning) and hourly new rapid update cycle forecasts (SINFONY-RUC-EPS) on the km-scale,
  • c) optimal combinations of Nowcasting and NWP ensemble forecasts in observation space, which constitute the seamless forecasts of the SINFONY. Gridded combined precipitation and reflectivity ensembles are targeted towards hydrologic warnings. Combined Nowcasting- and NWP cell object ensembles help evolve DWD's warning process for convective events towards a flexible "warn-on-objects",
  • d) systems for common Nowcasting and NWP verification of precipitation, reflectivity and objects. In particular the cell object based verification will provide new insights into the representation of deep convective cells in the model.

For b), new innovative and efficient forward operators for radar volume scans and visible satellite data (SEVIRI-VIS) enable
direct operational assimilation of these data in an LETKF framework. In late summer 2022, we finally could also activate the SEVIRI-VIS data in the daily RUC. Advanced model physics (2-moment bulk cloud mircophysics) contribute to an improved forecast of convective clouds. A stochastic PBL scheme has been developed, but is not yet in daily use.

For c), the SINFONY-RUC-EPS outputs simulated reflectivity volume scan ensembles of the entire German radar network every 5' online during its forecast runs. Ensembles of composites and cell object tracks are generated by the same compositing and cell detection- and tracking methods/software packages which are applied to the observations.

Considerable improvements in the RUC compared to the 2021 season lead to the situation that RUC starts to outperform Nowcasting already after about 1 hour lead time, both for areal Nowcasting and cell object Nowcasting. Life-cycles of simulated convective cells proved to be surprisingly realistic, although some problems in details of cell organization and cell size remain and need to be looked at.

Other presentations from SINFONY team members will give more details about particular SINFONY components.

How to cite: Blahak, U. and the Team SINFONY: Current status of SINFONY - the combination of Nowcasting and Numerical Weather Prediction on the convective scale at DWD, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-254, https://doi.org/10.5194/ems2023-254, 2023.

Onsite presentation
Meryl Wimmer, Loik Berre, Laurent Descamps, Laure Raynaud, and Yann Seity

AROME-EPS is the regional ensemble prediction system, operational at Météo-France, consisting of 12 perturbed non-hydrostatic forecasts with a 2.5km horizontal resolution. Model errors are currently represented with the Stochastic Perturbed Parametrization Tendency (SPPT) scheme. However, this method presents some disadvantages such as a difficult physical interpretation of its results. In order to overcome this drawback, a more physically-based approach of model error representation is considered, based on the perturbation of parameters from physical parametrization schemes. A two-step procedure is adopted to implement such a technique: a sensitivity analysis of the AROME model to some parameters is first performed, then an optimisation of perturbed parameters values is determined.

Following advice of parametrization experts, 21 parameters from 6 different physical and dynamical parametrizations, with uncertain values, have been selected. Sensitivity analyses, conducted on different seasons and using the Morris screening as well as Sobol' sensitivity indices, have led to reduce this list to a subset of eight parameters with a high influence on different weather-related variables forecasts. 

Several perturbed parameters techniques have then been set up and evaluated over long periods. They largely improve AROME-EPS performances for most near-surface variables including wind speed and accumulated precipitation. Different optimizations improving the statistical CRPS score have also been tested. Thus, a set of parameter values has been identified for each AROME-EPS member. This optimal perturbation parameter method is shown to significantly outperform the current SPPT scheme for several near-surface variables, with improvements of probabilistic scores up to 10%. Furthermore, restricting the perturbation to the eight most influential parameters has shown similar results as the version perturbing the full set of 21 parameters, suggesting a possible cheaper setting of weather prediction models.

How to cite: Wimmer, M., Berre, L., Descamps, L., Raynaud, L., and Seity, Y.: Development and evaluation of an "optimal" perturbed parameter approach to represent model error in the convective-scale AROME-EPS, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-310, https://doi.org/10.5194/ems2023-310, 2023.

Online presentation
Mariana Clare and Thomas Haiden

Machine learning techniques are increasingly used in weather forecasting, to either improve results from numerical weather prediction (NWP) models or emulate some aspects of the forward integration part of NWP models using a purely data-driven approach. So far most of the techniques presented have focused on producing deterministic forecasts, as many standard machine learning techniques lack the ability to express uncertainty.  

Whilst methods that attempt to fully emulate NWP models are both very complex and computationally expensive, in this talk I will show how much simpler and computationally cheaper techniques can be used to create a probabilistic forecast from a single global high-resolution forecast by post-processing. Specifically, I will focus on the relatively novel technique of a Bayesian Neural Network and show how it can predict the distribution of the forecast error relative to its own analysis. By adding these error distributions to the original forecast, we can create a probabilistic forecast. Time permitting, I will discuss how more advanced techniques can be applied to take these distributions and generate spatially consistent ensemble members. 

This methodology is particularly useful for NWP models at very high resolutions where running an ensemble is too computationally expensive and for machine learning approaches where no uncertainty information is available. In this talk, I will show how this methodology can be successfully applied to both ECMWF’s high-resolution forecast and a purely data-driven weather forecast model being run at ECMWF, for both the surface variable of 2m temperature and the atmospheric variable of Z500. These probabilistic forecasts have been verified using standard metrics and, in the case of the high-resolution numerical forecast, have as good as or better CRPS scores than the ECMWF ensemble forecast for the lead times tested. Moreover, these probabilistic forecasts are reliable with spread-skill ratios close to one. Hence, this novel machine learning post-processing technique has the potential to produce probabilistic forecasts that are valuable and useful to forecasters. 

How to cite: Clare, M. and Haiden, T.: Creating skillful and reliable probabilistic forecasts using machine learning, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-199, https://doi.org/10.5194/ems2023-199, 2023.

Online presentation
Andreas Beckert and Marc Rautenhaus

Ensemble simulations have become a standard in numerical weather prediction (NWP). However, ensemble simulations generate large amounts of data, and their comprehensive analysis remains challenging. We introduce a feature-based NWP ensemble analysis method based on the well-established conceptual model of atmospheric fronts. Recent developments in front detection techniques have enabled a reliable, robust, and objective identification of three-dimensional (3-D) frontal structures in NWP data.

We advance detection of individual front features towards front-feature-based time series analysis and ensemble clustering. We track 3-D fronts of a selected cyclone system to create time series of frontal attributes. These frontal attributes characterize properties of the tracked front, for example, the maximum strength of the temperature gradient across the frontal zone, the average slope of the 3-D frontal structure or associated upward motion. The obtained time series provide a compact overview of the development of front characteristics. For ensemble analysis, we generate such time series for the cyclone system as represented in the different ensemble members. Time series distance measures including dynamic time warping can then be utilized to analyze similarities and differences of frontal development in the ensemble, for example, for front-feature-based ensemble cluster analysis. Also, a time window similarity search enables users to select a specific event of interest (for example, a sudden increase in frontal strength) in one of the ensemble members and to search for similar events in other members.

Integrated in the 3-D visual analysis framework Met.3D, our approach facilitates a comprehensive analysis of the spatiotemporal development of 3-D atmospheric fronts and thus contributes to the challenge of rapidly analyzing large ensemble weather predictions.  

How to cite: Beckert, A. and Rautenhaus, M.: Ensemble analysis of spatiotemporal attributes derived from objectively identified three-dimensional atmospheric fronts, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-466, https://doi.org/10.5194/ems2023-466, 2023.

Onsite presentation
Hyuncheol Shin, Eun Jung Kim, Jong Im Park, Sug-gyeong Yun, Jong-Chul Ha, and Young-Cheol Kwon

The ensemble prediction system based on the Korean Integrated Model (KIM) has been in operation at Korea Meteorological Administration (KMA) since October 2021. KIM is KMA’s new generation global model which is based on the cubed sphere grid system and has 12km horizontal resolution and 91 vertical levels. It was developed over 9 years from 2011 to 2019 and operationally launched in April 2020. KIM-based ensemble forecast system consists of 50 perturbation members (25 members for long-range forecast) and 1 control member. Four-dimensional LETKF (Local Ensemble Transform 
Kalman Filter) with additive and RTPS inflation scheme is used to make initial perturbation.
The performance of KIM-based ensemble system was evaluated. It is generally more skillful compared to the KIM deterministic global model and shows similar performance with UM-based ensemble system which KMA is operating. An increase in the number of ensemble members results in an overall improvement in prediction performance, especially at higher latitudes. Details of results from KIM ensemble system and impacts of increased ensemble size will be discussed.
Multi-model ensemble is another type of ensemble prediction system KMA is operating. KMA’s multi-model ensemble prediction system that merges six global domain models(KIM, UM, ECMWF global and global ensemble models) is used for short and medium-range forecast. Multi-model ensemble mean shows better performance than the ECMWF ensemble model, which is the best member model. 
In addition, multi-model ensemble system that incorporate regional models in addition to global models is used for impact-based forecast of heat waves and cold waves.  Impact-based forecast for high risk level(`warning` and `alarm`) is improved by using multi-model ensemble system.

How to cite: Shin, H., Kim, E. J., Park, J. I., Yun, S., Ha, J.-C., and Kwon, Y.-C.: Various approaches using ensemble prediction system in KMA, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-415, https://doi.org/10.5194/ems2023-415, 2023.

Poster introduction
Coffee break
Chairperson: Estíbaliz Gascón
Onsite presentation
Irene Schicker, Petrina Papazek, Markus Dabernig, and Theresa Schellander-Gorgas

Subseasonal predictions are gaining more and more attention and importance in many applications, e.g. agriculture or energy&consumption predictions. To bridge the gap between those two temporal horizons and their drivers is, however, a challenge. Several attempts have been made in recent years to improve the numerical weather predictions but they to come at a high computation cost resulting in coarse spatial resolutions.  In the past decade, significant advances were made in improving the S2S and seasonal prediction using mainly numerical weather prediction models (NWP) and in some cases climate models for generating the predictions. Recently, the application of these models in real time forecasting through the S2S Real-Time Pilot Initiative (Robbins et al., 2020) was evaluated and is ongoing. There are, however, drawbacks. Computational costs for performing one forecast cycle are high (RAM, storage, ensemble for uncertainty) and limit the spatial, and to some extent temporal, resolution which are currently roughly 1.5° in spatial and at most 6-hourly in temporal resolution. Both resolutions are not sufficient for small scale renewable production sites. To overcome this, post-processing can be applied using statistical and machine leraning methods.

In this study, statistical (EPISODES, GMOS, SAMOS) and machine learning methods (U-net, random forest) are used to downscale and post-process the coarse subseasonal ensemble predictions for temperature and precipitation. The domain in centred on Austria with a spatial resolution of 1 km  using the INCA analysis as target fields. Evaluation against INCA and point observations show the skills of all methods and highlight the need for additional downscaling.

How to cite: Schicker, I., Papazek, P., Dabernig, M., and Schellander-Gorgas, T.: SSEA- Statistical and machine learning based post-processing for high-resolution subseasonal ensemble predictions, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-467, https://doi.org/10.5194/ems2023-467, 2023.

Onsite presentation
Ganglin Tian, Camille Le Coz, Anastase Alexandre Charantonis, Alexis Tantet, Naveen Goutham, and Riwal Plougonven

Wind power systems' maintenance, deployment, and management, as well as the balance between energy supply and demand, are highly dependent on wind speeds and their temporal variability. Wind speed prediction at the sub-seasonal time scale presents a challenge since the skill of surface wind prediction sharply declines after two weeks. Nevertheless, the predictability of large-scale variables is higher than that of surface variables at this time scale. Goutham et al. (2022) improved the skill of surface wind speed forecasts by downscaling 500 hPa geopotential height (Z500)t forecasts using redundancy analysis based on a linear framework. Leveraging their work, we investigate whether Convolutional Neural Networks (CNNs) can be used to further improve the skill of subseasonal wind speed predictions over Europe.


To answer this question, this study proposes a non-linear statistical sub-seasonal ensemble forecasting method for the boreal winter over Europe based on applying a supervised learning model to dynamic forecasts. Specifically, the proposed statistical CNN model regresses weekly-mean surface wind speeds from dynamic forecasts of Z500. The dynamical forecasts of surface wind speed from the European Centre for Medium-Range Weather Forecasts (ECMWF) and the statistical forecasts from Redundancy Analysis (RA, Goutham et al. 2022) are selected as benchmarks for comparison, known for their superiority to climatology in different domains, lead times, and assessment indicators. Initially, we access the skill of a deterministic version of the network, utilizing ERA5 reanalysis data (trained for 15 years and tested for 5 years) to regress wind speed from geopotential height, find that the network is, on average, more skillful than RA based on the root mean squared error. Subsequently, the same network (without retraining) is applied to the subseasonal predictions from the ECMWF  covering the boreal winters from 2015 to 2022, with the same variables. Several indicators, including the anomaly correlation coefficient, continuous ranked probability score, and corresponding skill scores, compare the skill of the statistical forecasts (RA and CNN) and the dynamical forecasts (ECMWF). This comparison aims to determine if the statistical CNN has superior skill in the primary wind energy-producing countries in Europe and whether different models exhibit specific spatiotemporal patterns of skill in the sub-seasonal range. This study also investigates the performance of statistical and dynamical wind speed forecasts under normal and extreme conditions by analyzing the probability density distribution of ensemble members at given areas. Our preliminary findings reveal that statistical forecasts exhibit superior skill in normal

How to cite: Tian, G., Le Coz, C., Alexandre Charantonis, A., Tantet, A., Goutham, N., and Plougonven, R.: Convolutional neural network downscaling to improve sub-seasonal wind-speed predictions in Europe, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-321, https://doi.org/10.5194/ems2023-321, 2023.

Onsite presentation
Alexandre Belleflamme, Klaus Goergen, Suad Hammoudeh, Niklas Wagner, and Stefan Kollet

The repeated and severe droughts that have affected central Europe over the last years (2018, 2019, 2020, 2022) have triggered the need for sub-seasonal to seasonal forecasts of subsurface water resources. Such long-term forecasts are needed in several sectors such as agriculture, forestry, and water resources, to allow for elaborating and implementing management strategies able to deal with the reduced water resources.

In this context, we have developed a high-resolution (0.6km) monitoring and forecasting system of the terrestrial water cycle with the integrated, physics-based hydrologic model ParFlow/CLM over Germany and the surrounding regions. This model setup simulates the surface and 3D subsurface state and fluxes down to 60m depth, thereby covering the variably saturated zone as well as the saturated zone of shallow groundwater bodies. We use the seasonal 50-member ensemble forecast SEAS from the European Centre for Medium-Range Weather Forecasts (ECMWF) as atmospheric forcing for ParFlow/CLM. We calculate these forecasts over the whole available lead time, i.e., seven months, at the beginning of each meteorological season (March, June, September, and December).

In this study, we analyse the ability for the ParFlow/CLM seasonal forecasts to predict the evolution of water resources, and in particular total subsurface storage, water table depth, and groundwater recharge, during the summer drought of 2022. To evaluate the forecasting skill, the 50-member ensemble seasonal forecasts are compared with a reference time series simulated by forcing ParFlow/CLM by the first 24h of each daily HRES deterministic forecast from ECMWF, which has been validated against observations in a previous study.

Finally, we provide some examples on how these seasonal forecasts can be synthesized to assess the risk of water resources depletion due to atmospheric drought conditions, thus providing useful information for society, and, in particular, the agricultural, forestry, and water management sectors.

How to cite: Belleflamme, A., Goergen, K., Hammoudeh, S., Wagner, N., and Kollet, S.: Assessment of forecast skill of seasonal forecasts with the hydrologic model ParFlow/CLM to predict subsurface water resources under atmospheric drought conditions in central Europe, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-348, https://doi.org/10.5194/ems2023-348, 2023.

Onsite presentation
Damien Specq, Shan Li, Lauriane Batté, Christian Viel, and Frédéric Gayrard

Seasonal prediction uses ensemble forecasting to sample the distribution of possible climate outcomes in the upcoming term given the slowly-varying constraints on the atmosphere. However, translating the members’ distribution of a seasonal forecast into meaningful information is a challenge climate services are often faced with. When a large ensemble spread makes the forecast difficult to interpret, highlighting the competing signals from which the uncertainty arises may bear added value to end users. In order to do so, we present an approach to extract alternative seasonal forecast scenarios over Europe (in temperature, precipitation and atmospheric circulation) from ensemble seasonal forecasts. The aim of the scenarios is to refine the ensemble analysis beyond the usual forecast products (e.g ensemble mean, tercile probabilities), and to provide additional guidance for preparation of the seasonal forecast bulletins routinely issued at Météo-France.

The seasonal forecast scenarios are determined with a hierarchical clustering of the ensemble members, based on their forecast temperature at 2-m (T2m). The dissimilarity between two members is defined from the spatial correlation between their respective maps of T2m anomalies – relative to model climatology – over a European domain (29.5°W-40.5°E; 30.5°N-70.5°N, land grid points only). The subsequent dissimilarity matrix across the ensemble feeds the clustering algorithm that groups members into clusters eventually defining the scenarios. The seasonal outcomes corresponding to these scenarios are then described through several diagnostics, e.g composites on sensible climate variables (T2m, precipitation), composites on atmospheric circulation variables (Z500, V200), and analysis through modes of variability and weather regimes. In addition, we provide a description of how scenarios diverge in the course of forecast integration and identify teleconnections related to each scenario. Finally, we also assess the skill of the seasonal forecasts assuming that only the subset of members representing the most likely scenario is retained.

This methodology has been implemented to the Copernicus Climate Change Services (C3S) real-time seasonal forecasts across the past year for experimental purposes, and it is shown to be a relevant complement for the preparation of the Météo-France operational seasonal bulletins.

How to cite: Specq, D., Li, S., Batté, L., Viel, C., and Gayrard, F.: Multiple scenarios of climate anomalies over Europe in ensemble seasonal forecasts, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-31, https://doi.org/10.5194/ems2023-31, 2023.

Onsite presentation
Shan Li, Damien Specq, Lauriane Batté, and Christian Viel

The current Météo-France seasonal prediction system (MF System 8) has 25 members for hindcast from 1993 to 2016 and 51 members for real-time forecast. In order to investigate the benefits of increasing the ensemble size within our Copernicus Climate Change Services (C3S) seasonal prediction contract, we extended the system 8 hindcast to 51 members for the four main start dates (February, May, August, November). We compare the forecast skill between the official 25-member hindcast and the 51-member extended hindcast.

We focus on the European region at the lead-time 1 for the next trimester. To describe the performance of forecasts, we use correlation and relative operating characteristic (ROC) on the mean 2-meter temperature (T2M) and the mean precipitation (RR) averaged over Europe. Similarly, we also evaluate the forecast of modes of variability (East Atlantic, North Atlantic Oscillation, and Scandinavian Blocking) which impact the European climate.

The scores with 51 members are similar and not necessarily better than with 25 members. Moreover, we use 1000 random draws of 25 members out of 51 to determine the uncertainty of the official forecast scores. These scores can be at the edge of the confidence interval, while the 51-member scores are close to the median of the 1000 random draws. We did the same analysis for different regions that there is less uncertainty on the scores in the Tropics (e.g. Northeast Brazil) than in the mid-latitudes (e.g. Europe). These results suggest that it is not necessary to increase the ensemble size for verification of the seasonal forecasts beyond the available 25 members.

How to cite: Li, S., Specq, D., Batté, L., and Viel, C.: Benefits of increasing the reforecast ensemble size of the Météo-France seasonal prediction system, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-32, https://doi.org/10.5194/ems2023-32, 2023.

Online presentation
Ivana Aleksovska

Numerical weather prediction models have systematic errors and biases, which can be reduced through statistical correction methods. The most intuitive approach to correcting weather forecasts is to establish a statistical relationship between the forecast and the corresponding observations. Once the relationship is established, it can be used to correct future forecasts. These approaches, also known as statistical adaptation, are used on a daily basis to improve the quality of operational forecasts. Many methods have been proposed in the literature to calibrate ensemble forecasts, and these can be divided into two main groups: parametric (that has a hypothesis on the underlying distribution) and non-parametric methods. One of the most widespread and used methods of the parametric group is the so-called Ensemble Model Output Statistics (EMOS).

We show the performance of EMOS for the post-processing of probabilistic temperature forecasts at 2m. This method assumes that the underlying distribution is Gaussian (in the case of 2m T), and the parameters to be estimated are therefore the mean and the standard deviation. First results were obtained using the ECMWF operational forecast ensemble against SYNOP observations. Further studies were carried out to investigate the reliability of the estimated parameters using not only the raw ensemble data, but also the ECMWF deterministic operational forecast HRES and the control member. The results showed an improvement, especially for the shorter lead-times. The improvement was measured using weather scores for forecast verification and performance: bias, CRPS (Continuous Ranked Probability Score) and CRPSS (Continuous Ranked Probability Skill Score). These post-processed forecasts will serve as a reference for future studies.

How to cite: Aleksovska, I.: Post-processing ensemble forecasts using Ensemble Model Output Statistics (EMOS), EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-619, https://doi.org/10.5194/ems2023-619, 2023.

Posters: Tue, 5 Sep, 16:00–17:15 | Poster area 'Day room'

Display time: Mon, 4 Sep 09:00–Wed, 6 Sep 09:00
Chairperson: Andrea Montani
Camille Le Coz, Alexis Tantet, Rémi Flamary, and Riwal Plougonven

Multi-model ensemble (MME) methods, i.e. combining ensemble forecasts produced by different models, have been shown to improve the skill of prediction at different scales. One important research question is how to best construct such multi-model ensemble. This is one of the challenges addressed by the subseasonal-to-seasonal (S2S) project for the subseasonal scale, and also the focus of this study. To answer this question, we compare two methods based on barycenters.

The main idea is to consider the ensemble forecasts as discrete probability distributions, and to use a barycentre to combine them. We compare two barycenters based on different distances, the L2 and the Wasserstein distance. Applying an L2-barycentre on ensemble forecasts is equivalent to concatenating their members, i.e. to the well-known MME method known as “pooling”. The Wasserstein distance corresponds to the cost of the optimal transport between two distributions and has interesting properties in the distribution space. The two methods potentially lead to very different multi-model ensembles. The barycenters also allow us to attribute weights to the models. These weights are learned from the data (using cross-validation on the forecasts) in order to optimize the skill of the barycenter-ensembles.

In order to investigate the benefits of these two multi-model ensemble methods, both methods are applied to the combination of two models from the S2S forecasts, namely the European Centre Medium-Range Weather Forecasts (ECMWF) and the National Centers for Environmental Prediction (NCEP) models. The skill of the two barycenter-ensembles are evaluated for the prediction of weekly 2m-temperature over Europe for seven winters with respect to different metrics. Although the ECMWF model has an overall better performance than NCEP, the barycenter-ensembles are generally able to outperform both. However, the best ensemble depends on the chosen metric and on the location. Focusing on the combination of two models has allowed us to investigate the impact of the model’s weights on the performance of the barycenters. It also provides encouraging results for the next step, the combination of several models.

How to cite: Le Coz, C., Tantet, A., Flamary, R., and Plougonven, R.: A barycenter-based approach for the multi-model combination of ensemble forecasts: an application to the sub-seasonal forecasts of 2m-temperature over Europe, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-483, https://doi.org/10.5194/ems2023-483, 2023.

Sug-gyeong Yun, Hyun-Cheol Shin, Eun Jung Kim, Jong Im Park, Jong-Chul Ha, Young Cheol Kwon, and KieWoung Lee

  The Korea Meteorological Administration(KMA) is producing an impact-based forecast data based on both the deterministic forecast and ensemble forecast for heat waves (HW) and cold waves (CW). Ensemble prediction system for impact-based forecast is Multi-Model ensemble system which integrates UM(global, global ensemble, local, and local ensemble models), ECMWF(global and global ensemble models) and KIM(Korean Integrated Model) global model(Hereafter, impact-based forecast  based on the deterministic forecast and Multi-Model ensemble system are called `IMPC` and ‘MEPS’, respectively.). MEPS determine the risk level by using the probability of occurrence of abnormal temperatures on the Korean Peninsula. Once maximum perceived temperature(HW) or lowest temperature(CW) from all 93 Multi-Model Ensemble members were extracted, their probability distribution was determined by using a Generalized Extreme Value (GEV) distribution.
  The performance of MEPS for HW was compared with IMPC for July and August 2022. Verification was conducted by evaluating how well impact-based forecast level(safe, concern, caution, warning, alarm) was matched to the observed risk level in 175 regions. 
  As a result of verification it was found that IMPC forecasts ‘concern’ more frequently and MEPS forecasts ‘caution’ more frequently. In July, IMPC's prediction performance is excellent for ‘safe’ and ‘concern’ level and MEPS is excellent at a ‘caution’ and ‘warning’. IMPC underestimates ‘caution’ and ‘warning’ while MEPS tends to overestimate ‘safe’ and ‘concern’. In August, ETS score of MEPS is excellent for ‘safe’ levels as well as ‘caution’ and ‘warning’ levels. In case studies, there is many cases in which MEPS detected HW better than IMPC when a high-level heat wave was observed on the Korean Peninsula. Overall, MEPS is expected to be a good reference data in the impact-based forecast where predictive ability for high risk levels is important. 
  The MEPS guidance uses only daily temperatures, but according to KMA’s forecast guideline, HW is defined a phenomenon in which a high temperature lasts for more than 2 days. By reflecting these conditions, future guidance needs to be improved. In the HW guidance, the risk level re-established by considering whether the risk level lasts longer tha two days improved the predictive performance of ‘safe’ and ‘concern’, and the CSI (Critical Success Index) and ETS increased at all risk level.

How to cite: Yun, S., Shin, H.-C., Kim, E. J., Park, J. I., Ha, J.-C., Kwon, Y. C., and Lee, K.: Verification and Improvement of Impact-based Forecast Using Multi Model Ensemble, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-397, https://doi.org/10.5194/ems2023-397, 2023.

Miloslav Belorid, Bu-yo Kim, Haejung Koo, and Joo Wan Cha

In countries where drought is a serious threat to agriculture, water resources and increases risk of wildfires, there has been an increasing interest in using weather modification techniques to improve local precipitation.  Despite the fact that cloud seeding is used in many places around the world, there is still not a clear consensus on its effectiveness. Various evaluation methods, including aircraft and ground-based measurements, remote sensing, statistical analysis, and numerical simulation have been widely used to evaluate the effect of cloud seeding. While each of these methods has its own benefits, they also come with limitations. For example, numerical models can simulate cases with and without cloud seeding and these scenarios can be then compared to determine how much rainfall increases after the seeding. Predictions provided by such scenarios can help with decision-making before conducting a cloud seeding experiment. The downside of numerical simulations is the presence of both systematic and random errors that originate from uncertainties initial conditions and numerical approximations. Ensemble forecasts can capture some of these uncertainties and provide a range of possible outcomes. The main goal of this study is to explore the potential of an ensemble forecasting system in evaluating the efficacy of cloud seeding. We used a limited-area ensemble forecasting system which is based on Met Office Unified Model coupled with Weather Research and Forecasting Model (WRF). The initial conditions of 13 ensemble members were created by downscaling of global ensemble model that was perturbed using the Ensemble transform Kalman filter. The WRF model was used to downscale the ensemble members to a finer resolution and simulate the seeding effect using an algorithm that was added to the Morrison microphysics scheme. As study cases, we utilized several cloud seeding experiments that were conducted during the weather modification campaign of 2022 and 2023 by the National Institute of Meteorological Sciences and the Korea Meteorological Administration. Ground-based hygroscopic cloud seeding was conducted in the mountainous region of South Korea using calcium chloride flares. Simulations were conducted for scenarios with and without seeding, and the difference in rainfall we assessed using ensemble mean, ensemble spread, and probabilities of various rainfall increment thresholds. The overall ensemble performance in rain predictability of the ensemble forecasts was evaluated using commonly used techniques, including the Brier score, reliability diagrams and ROC curves.

Acknowledgments: This work was funded by the Korea Meteorological Administration Research and Development Program “Research on Weather Modification and Cloud Physics” under Grant (KMA2018-00224).

How to cite: Belorid, M., Kim, B., Koo, H., and Cha, J. W.: Application of Ensemble Forecasts in Predicting and Assessing the Effectiveness of Rain Enhancement using Cloud Seeding, EMS Annual Meeting 2023, Bratislava, Slovakia, 4–8 Sep 2023, EMS2023-191, https://doi.org/10.5194/ems2023-191, 2023.

Additional speaker

  • Anders Doksæter Sivle, MET, Norway