Displays

NP5.4

Statistical post-processing techniques for weather, climate, and hydrological forecasts are powerful approaches to compensate for effects of errors in model structure or initial conditions, and to calibrate inaccurately dispersed ensembles. These techniques are now an integral part of many forecasting suites and are used in many end-user applications such as wind energy production or flood warning systems. Many of these techniques are flourishing in the statistical, meteorological, climatological, hydrological, and engineering communities. The methods range in complexity from simple bias correction up to very sophisticated distribution-adjusting techniques that take into account correlations among the prognostic variables.

At the same time, a lot of efforts are put in combining multiple forecasting sources in order to get reliable and seamless forecasts on time ranges from minutes to weeks. Such blending techniques are currently developed in many meteorological centers.

In this session, we invite papers dealing with both theoretical developments in statistical post-processing and evaluation of their performances in different practical applications oriented toward environmental predictions, papers dealing with the problem of combining or blending different types of forecasts in order to improve reliability from very short to long time scales.

Share:
Co-organized by AS5/CL5/HS4
Convener: Stéphane Vannitsem | Co-conveners: Stephan HemriECSECS, Maxime TaillardatECSECS, Daniel S. Wilks
Displays
| Attendance Fri, 08 May, 16:15–18:00 (CEST)

Files for download

Session materials Download all presentations (73MB)

Chat time: Friday, 8 May 2020, 16:15–18:00

D2463 |
EGU2020-2588
Moritz N. Lang, Sebastian Lerch, Georg J. Mayr, Thorsten Simon, Reto Stauffer, and Achim Zeileis

Non-homogeneous regression is a frequently-used post-processing method for increasing the predictive skill of probabilistic ensemble weather forecasts. To adjust for seasonally varying error characteristics between ensemble forecasts and corresponding observations, different time-adaptive training schemes, including the classical sliding training window, have been developed for non-homogeneous regression. This study compares three such training approaches with the sliding-window approach for the application of post-processing near-surface air temperature forecasts across Central Europe. The predictive performance is evaluated conditional on three different groups of stations located in plains, in mountain foreland, and within mountainous terrain, as well as on a specific change in the ensemble forecast system of the European Centre for Medium-Range Weather Forecasts (ECMWF) used as input for the post-processing.

The results show that time-adaptive training schemes using data over multiple years stabilize the temporal evolution of the coefficient estimates, yielding an increased predictive performance for all station types tested compared to the classical sliding-window approach based on the most recent days only. While this may not be surprising under fully stable model conditions, it is shown that "remembering the past" from multiple years of training data is typically also superior to the classical sliding-window when the ensemble prediction system is affected by certain model changes. Thus, reducing the variance of the non-homogeneous regression estimates due to increased training data appears to be more important than reducing its bias by adapting rapidly to the most current training data only.

How to cite: Lang, M. N., Lerch, S., Mayr, G. J., Simon, T., Stauffer, R., and Zeileis, A.: Remember the past: A comparison of time-adaptive training schemes for non-homogeneous regression, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-2588, https://doi.org/10.5194/egusphere-egu2020-2588, 2020.

D2464 |
EGU2020-17465
Nousu Jari-Pekka, Matthieu Lafaysse, Guillaume Evin, Matthieu Vernay, Joseph Bellier, Bruno Joly, Maxime Taillardat, and Michaël Zamo

Forecasting the height of new snow (HS) is essential for avalanche hazard survey, road and ski resorts management, tourism attractiveness, etc. Meteo-France operates the PEARP-S2M probabilistic forecasting system including 35 members of the PEARP Numerical Weather Prediction system, the SAFRAN downscaling tool refining the elevation resolution in mountains, and the Crocus snowpack model representing the main physical processes in the snowpack (compaction, melting, etc.). It provides better HS forecasts than direct NWP diagnostics but exhibits significant biases and underdispersion. Therefore, a post-processing is required to be able to provide automatic forecasting products of HS from this system.

For that purpose, we compare the skill of two statistical methods (Nonhomogeneous Regression with a Censored Shifted Gamma distribution and Quantile Regression Forest), two predictor datasets for training (22-year reforecast with some discrepancies with the operational system or 3-year real time forecasts similar to the operational system) and two spatial scales of post-processing (local scale or 1000 km² regional scale).

The improvement relative to the raw forecasts is similar at both spatial scales. Thus, the regional validity of post-processing does not restrict the application at points with observations. The impact of the training dataset depends on lead time and on the evaluation criteria. The long-term reforecast improves the reliability of severe snowfall but leads to overdispersion due to a discrepancy with the initial perturbations used in the operational system. Finally, thanks to a larger number of predictors, the Quantile Regression Forest allows an improvement of forecasts for specific cases when the the rain-snow transition elevation is overestimated by the raw forecasts.

These conclusions help to choose an optimal post-processing configuration for automatic forecasts of the height of new snow and encourage the atmospheric modelling teams to develop long reforecasts as homogenous as possible with the operational systems.

How to cite: Jari-Pekka, N., Lafaysse, M., Evin, G., Vernay, M., Bellier, J., Joly, B., Taillardat, M., and Zamo, M.: Impact of the statistical method, training dataset, and spatial scale of post-processing to adjust ensemble forecasts of the height of new snow, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-17465, https://doi.org/10.5194/egusphere-egu2020-17465, 2020.

D2465 |
EGU2020-9698
Julian Steinheuer and Petra Friederichs
Wind and gust statistics at the hub height of a wind turbine are important parameters for the planning in the renewable energy sector. However, reanalyses based on numerical weather prediction models typically only give estimates for wind gusts at the standard measurement height of 10 m above the land surface. We present here a statistical post-processing that gives a conditional distribution for hourly peak wind speeds as a function of height. The conditioning variables are provided by the regional reanalysis COSMO-REA6. The post-processing is developed on the basis of observations of the peak wind speed in five vertical layers between 10 m and 250 m of the Hamburg Weather Mast. The statistical post-processing is based on a censored generalized extreme value (cGEV) distribution with non-stationary parameters. To select the most meaningful variables we use a least absolute shrinkage and selection operator. The vertical variation of the cGEV parameters is approximated using Legendre polynomials, allowing gust prediction at any desired height within the training range. Furthermore, the Pickands dependence function is used to investigate dependencies between gusts at different heights. The main predictors are the 10 m gust diagnosis, the barotropic and baroclinic modes of absolute horizontal wind speed, the mean absolute horizontal wind in 700 hPa, the surface pressure tendency and the lifted index. Proper scores show improvements of up to 60 %, especially at higher vertical levels when compared to climatology. The post-processing model with a Legendre approximation is able to provide reliable predictions of gust statistics at unobserved intermediate levels. The strength of the dependence between the gusts at different levels is not stationary and strongly modulated by the vertical stability of the atmosphere.

How to cite: Steinheuer, J. and Friederichs, P.: Vertical profiles of wind gust statistics from a regional reanalysis using multivariate extreme value theory, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-9698, https://doi.org/10.5194/egusphere-egu2020-9698, 2020.

D2466 |
EGU2020-11422
Jonas Bhend, Christoph Spirig, Max Hürlimann, Lionel Moret, and Mark Liniger

Weather forecasts have been steadily improving in quality over the last decades. These ongoing improvements are due to advances in numerical weather prediction (NWP) and the advent of ever more powerful supercomputers that allow simulating future weather and its uncertainty with increasing resolution and using ensemble approaches. Such physics-based computer models, however, are not free of systematic errors. Statistical postprocessing can be used to calibrate NWP forecasts to further improve forecast quality and better exploit the available information. Here we present results from several explorative deep learning studies using artificial neural networks (ANN) to calibrate high resolution forecasts of temperature, precipitation, wind, and cloud cover in Switzerland. These first attempts at ANN-based postprocessing help us to understand the strengths and weaknesses of machine learning and are the basis to build more complex and comprehensive statistical models accounting for local effects in complex terrain such as the Swiss Alps. In all cases, ANN leads to significant improvements over the direct NWP output. While the improvement is comparable in magnitude with improvements achieved with conventional postprocessing approaches, ANN-based postprocessing is easier to generalize in space for a calibration of forecasts also at unobserved sites. In addition to the results of the postprocessing, we will also discuss the lessons learned so far in using machine learning for this particular problem.

How to cite: Bhend, J., Spirig, C., Hürlimann, M., Moret, L., and Liniger, M.: Using neural networks for postprocessing of numerical weather predictions in complex terrain, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-11422, https://doi.org/10.5194/egusphere-egu2020-11422, 2020.

D2467 |
EGU2020-16849
Maurice Schmeits, Simon Veldkamp, and Kirien Whan

Current statistical post-processing methods for providing a probabilistic forecast are not capable of using full spatial patterns from the numerical weather prediction (NWP) model output. Recent developments in deep learning (notably convolutional neural networks) have made it possible to use large gridded input data sets. This could potentially be useful in statistical post-processing, since it allows us to use more spatial information.

In this study we consider wind speed forecasts for 48 hours ahead, as provided by KNMI's Harmonie-Arome model. Convolutional neural networks, fully connected neural networks and quantile regression forests are used to obtain probabilistic wind speed forecasts. Comparing these methods shows that convolutional neural networks are more skillful than the other methods, especially for medium to higher wind speeds.

How to cite: Schmeits, M., Veldkamp, S., and Whan, K.: Statistical post-processing of wind speed forecasts using convolutional neural networks, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-16849, https://doi.org/10.5194/egusphere-egu2020-16849, 2020.

D2468 |
EGU2020-18325
Florian Dupuy, Olivier Mestre, and Léo Pfitzner

Cloud cover is a crucial information for many applications such as planning land observation missions from space. However, cloud cover remains a challenging variable to forecast, and Numerical Weather Prediction (NWP) models suffer from significant biases, hence justifying the use of statistical post-processing techniques. In our application, the ground truth is a gridded cloud cover product derived from satellite observations over Europe, and predictors are spatial fields of various variables produced by ARPEGE (Météo-France global NWP) at the corresponding lead time.

In this study, ARPEGE cloud cover is post-processed using a convolutional neural network (CNN). CNN is the most popular machine learning tool to deal with images. In our case, CNN allows to integrate spatial information contained in NWP outputs. We show that a simple U-Net architecture produces significant improvements over Europe. Compared to the raw ARPEGE forecasts, MAE drops from 25.1 % to 17.8 % and RMSE decreases from 37.0 % to 31.6 %. Considering specific needs for earth observation, special interest was put on forecasts with low cloud cover conditions (< 10 %). For this particular nebulosity class, we show that hit rate jumps from 40.6 to 70.7 (which is the order of magnitude of what can be achieved using classical machine learning algorithms such as random forests) while false alarm decreases from 38.2 to 29.9. This is an excellent result, since improving hit rates by means of random forests usually also results in a slight increase of false alarms.

How to cite: Dupuy, F., Mestre, O., and Pfitzner, L.: ARPEGE cloud cover forecast post-processing with convolutional neural network, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-18325, https://doi.org/10.5194/egusphere-egu2020-18325, 2020.

D2469 |
EGU2020-11478
Sam Allen, Chris Ferro, and Frank Kwasniok

Raw output from deterministic numerical weather prediction models is typically subject to systematic biases. Although ensemble forecasts provide invaluable information regarding the uncertainty in a prediction, they themselves often misrepresent the weather that occurs. Given their widespread use, the need for high-quality wind speed forecasts is well-documented. Several statistical approaches have therefore been proposed to recalibrate ensembles of wind speed forecasts, including a heteroscedastic censored regression approach. An extension to this method that utilises the prevailing atmospheric flow is implemented here in a quasigeostrophic simulation study and on reforecast data. It is hoped that this regime-dependent framework can alleviate errors owing to changes in the synoptic-scale atmospheric state. When the wind speed strongly depends on the underlying weather regime, the resulting forecasts have the potential to provide substantial improvements in skill upon conventional post-processing techniques. This is particularly pertinent at longer lead times, where there is more improvement to be gained upon current methods, and in weather regimes associated with wind speeds that differ greatly from climatology. In order to realise this potential, however, an accurate prediction of the future atmospheric regime is required.

How to cite: Allen, S., Ferro, C., and Kwasniok, F.: Regime-dependent statistical post-processing of wind speed forecasts, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-11478, https://doi.org/10.5194/egusphere-egu2020-11478, 2020.

D2470 |
EGU2020-315
Jon Saenz, Sheila Carreno-Madinabeitia, Ganix Esnaola, Santos J. González-Rojí, Gabriel Ibarra-Berastegi, and Alain Ulazia

A new diagram is proposed for the verification of vector quantities generated by individual or multiple models against a set of observations. It has been designed with the idea of extending the Taylor diagram to two-dimensional vector such as currents, wind velocity, or horizontal fluxes of water vapour, salinity, energy and other geophysical variables. The diagram is based on a principal component analysis of the two-dimensional structure of the mean squared error matrix between model and observations. This matrix is separated in two parts corresponding to the bias and the relative rotation of the empirical orthogonal functions of the data. We test the performance of this new diagram identifying the differences amongst a reference dataset and different model outputs using examples wind velocities, current, vertically integrated moisture transport and wave energy flux time series. An alternative setup is also proposed with an application to the time-averaged spatial field of surface wind velocity in the Northern and Southern Hemispheres according to different reanalyses and realizations of an ensemble of CMIP5 models. The examples of the use of the Sailor diagram show that it is a tool which helps identifying errors due to the bias or the orientation of the simulated vector time series or fields. An implementation of the algorithm in form of an R package (sailoR) is already publicly available from the CRAN repository, and besides the ability to plot the individual components of the error matrix, functions in the package also allow to easily retrieve the individual components of the mean squared error.

How to cite: Saenz, J., Carreno-Madinabeitia, S., Esnaola, G., González-Rojí, S. J., Ibarra-Berastegi, G., and Ulazia, A.: The Sailor-diagram. An extension of the Taylor diagram to two-dimensional variables for verification of model data., EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-315, https://doi.org/10.5194/egusphere-egu2020-315, 2020.

D2471 |
EGU2020-927
Swati Singh, Kaustubh Salvi, Subimal Ghosh, and Subhankar Karmakar

The downscaling approaches: Statistical and Dynamic, developed for regional climate predictions, have both advantages and limitations. The statistical downscaling is computationally inexpensive but suffers from the violation of the assumption of stationarity in statistical (predictor-predictand) relationship. The dynamical downscaling is assumed to take care of stationarity but suffers from the biases associated with various sources.  Here we propose a joint approach of both the methods by applying statistical methods: bias correction & statistical downscaling to Coordinated Regional Climate Downscaling Experiment (CORDEX) evaluation runs. The evaluation runs are considered as perfect simulations of CORDEX Regional Climate Models (RCMs) with the boundary conditions by ERA-Interim reanalysis data. The statistical methods are also applied to ERA-Interim reanalysis data and compared with observation data for Indian Summer Monsoon characteristics. We evaluate the ability of statistical methods under the non-stationary environment by taking the difference of years close to extreme future runs (RCP8.5) as warmer years and preindustrial runs as cooler years. We find statistical downscaling of CORDEX evaluation runs shows skill in reproducing the signal of non-stationarity. The study can be extended methods by applying statistical downscaling to CORDEX RCMs with the CMIP5 boundary conditions. 

How to cite: Singh, S., Salvi, K., Ghosh, S., and Karmakar, S.: Fidelity of CORDEX Evaluation runs under Non-stationary climate, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-927, https://doi.org/10.5194/egusphere-egu2020-927, 2020.

D2472 |
EGU2020-4039
Zhaolu Hou, Bin Zuo, Shaoqing Zhang, Fei Huang, Ruiqiang Ding, Wansuo Duan, and Jianping Li

Numerical forecasts always have associated errors. Analogue correction methods combine numerical simulations with statistical analyses to reduce model forecast errors. However, identifying appropriate analogues remains a challenging task. Here, we use the Local Dynamical Analog (LDA) method to locate analogues and correct model forecast errors. As an example, an ENSO model forecast error correction experiment confirms that the LDA method locates more dynamical analogues of states of interest and better corrects forecast errors than do other methods. This is because the LDA method ensures similarity of the initial states and the evolution of both states. In addition, the LDA method can be applied using a scalar time series, which reduces the complexity of the dynamical system. Model forecast error correction using the LDA method provides a new approach to correcting state-dependent model errors and can be readily integrated with other advanced models.

How to cite: Hou, Z., Zuo, B., Zhang, S., Huang, F., Ding, R., Duan, W., and Li, J.: Model forecast error correction based on the Local Dynamical Analog method: an example application to the ENSO forecast, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-4039, https://doi.org/10.5194/egusphere-egu2020-4039, 2020.

D2473 |
EGU2020-5664
Jonathan Demaeyer and Stéphane Vannitsem

For most statistical post-processing schemes used to correct weather forecasts, changes to the forecast model induce a considerable reforcasting effort. We present a new approach based on response theory to cope with slight model change. In this framework, the model change is seen as a perturbation of the original forecast model. The response theory allows then to evaluate the variation induced on the averages involved in the statistical post-processing, provided that the magnitude of this perturbation is not too large.

This approach is studied in the context of a simple quasi-geostrophic model. It provides a proof-of-concept of the potential performances of response theory in a chaotic system. The parameters of the statistical post-processing used - an Error-in-Variables Model Output Statistics (EVMOS) - are appropriately corrected when facing a model change. The potential application in a more operational environment is also discussed.

How to cite: Demaeyer, J. and Vannitsem, S.: Correcting for Model Changes in Statistical Post-Processing - An approach based on Response Theory, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-5664, https://doi.org/10.5194/egusphere-egu2020-5664, 2020.

D2474 |
EGU2020-6242
Juwon Kim, Hae-Jin Kong, and Hyuncheol Shin

Multi-model ensemble using statistical post-processing is one of the methods to provide the impact of uncertainties of the Numerical Weather Prediction (NWP) models, with low cost and better accuracy for extreme weather forecasts. Extreme weather events such as heat/cold waves, windstorms, and heavy rainfall result in severe damage in human life and properties. However, the performance of the NWP models, particularly, heavy rain forecast is still low due to the intermittent and non-Gaussian properties. The light rain tends to be overestimated and the strong rain tends to be underestimated averagely on the NWP models. Thus the multi-model ensemble using statistical post-processing is activated to correct the discrepancies between the observation and the model intensity of precipitation.
The aim of this study is to provide the improvement of precipitation forecasts in probabilistic and deterministic aspects using a multi-model ensemble method with more weights on the less error and without any bias correction. Six types of models, namely, Local Data assimilation and Prediction System (LDPS), Local ENsemble System (LENS), Global Data assimilation and Prediction System (GDPS), Ensemble Prediction System-Global (EPSG) of Korea Meteorological Administration (KMA), the single and ensemble models of European Centre for Medium-Range Weather Forecasts (ECMWF), are used to blend. The preliminary results of the multi-model ensemble show similar results to the ECMWF ensemble mean in deterministic for 3-hourly accumulated precipitation over the East Asia and the middle of the performance among individual models in probabilistic over the South Korea. More details of the methodology, results, and improvements will be discussed in the presentation.

How to cite: Kim, J., Kong, H.-J., and Shin, H.: Probability and deterministic amount of precipitation on the multi-model ensemble, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-6242, https://doi.org/10.5194/egusphere-egu2020-6242, 2020.

D2475 |
EGU2020-6811
Iris Odak Plenkovic, Suzana Panezic, and Endi Keresturi

Even the state-of-the-art mesoscale models exhibit noteworthy errors, especially in the complex terrain. Therefore, it is useful to include post-processing methods in the forecasting system to further reduce starting model errors at locations where measurements are available. 
The analog-based method (ABM) is a point-based post-processing approach which consists of two steps. The first step is to find the most similar past numerical weather predictions (analogs) over several variables (predictors) and the second is to form an analog ensemble (AnEn) out of the corresponding observations. 
The ABM is thoroughly tested using the wind speed NWP input, focusing on the complex terrain. Since August 2019 it is used in a test operational mode at Croatian Meteorological and Hydrological Service. The setup includes 15 members wind speed, wind gusts and temperature ensemble predictions for approximately 50 stations using the 2-year training dataset. The preliminary results show that the ABM implementation is successful, reducing the error and improving the skill of the raw model. Additionally, it is found that the ABM predictions of wind speed and gusts optimally need more predictors than the temperature predictions. Finally, the forecasting system shows the best result in the coastal region for the temperature predictions, while the best results for the wind speed are achieved in the nearly flat continental terrain situated more inland. 

How to cite: Odak Plenkovic, I., Panezic, S., and Keresturi, E.: Evaluation of the analog-based method for the operational implementation in Croatia, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-6811, https://doi.org/10.5194/egusphere-egu2020-6811, 2020.

D2476 |
EGU2020-7401
Madlen Peter, Alexander Pasternack, and Henning Rust

In weather and climate science statistical modeling is applied for manifold problems. Due to the increasing number of input variables, overfitting can easily deteriorate the performance for model predictions.  In order to avoid this, it is often meaningful to apply model selection approaches. Since conventional approaches can be very time-consuming especially for many predictors, we are using the boosting approach, which combines model selection and parameter estimation.  This iterative algorithm identifies and updates in each step only the most important coefficient, such that in the end most important predictor variables have non-zero coefficients and less relevant variables are ignored.
Boosting has been originally developed for classification problems but has also been extended and used for other applications; i.a. non-homogeneous gaussian regression. Based on the  non-homogeneous boosting proposed by Messner et al. (2016), which is used to model mean and variance of a forecast distribution simultaneously, we have developed a boosting algorithm for a non-stationary Generalized Extreme Value distribution (GEV). Thus, it is possible to identify the most relevant predictor variables for location, scale and shape parameter concurrently. We apply this algorithm to various toy model simulations to assess the effect of this novel approach.

How to cite: Peter, M., Pasternack, A., and Rust, H.: A boosting algorithm for Generalized Extreme Value distributions, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-7401, https://doi.org/10.5194/egusphere-egu2020-7401, 2020.

D2477 |
EGU2020-7804
Maxime Taillardat and Olivier Mestre

Statistical post-processing of ensemble forecasts, from simple linear regressions to more sophisticated techniques, is now a well-known procedure in order to correct biased and misdispersed ensemble weather predictions. However, practical applications in National Weather Services is still in its infancy compared to deterministic post-processing. This paper presents two different applications of ensemble post-processing using machine learning at an industrial scale. The first is a station-based post-processing of surface temperature in a medium resolution ensemble system. The second is a gridded post-processing of hourly rainfall amounts in a high resolution ensemble prediction system. The techniques used rely on quantile regression forests (QRF) and ensemble copula coupling (ECC), chosen for their robustness and simplicity of training whatever the variable subject to calibration.

Moreover, some variants of classical techniques used such as QRF or ECC have been developed in order to adjust to operational constraints. A forecast anomaly-based QRF is used for temperature for a better prediction of cold and heat waves. A variant of ECC for hourly rainfall is built, accounting for more realistic longer rainfall accumulations. It is shown that forecast quality as well as forecast value is improved compared to the raw ensemble. At last, comments about model size and computation time are made.

How to cite: Taillardat, M. and Mestre, O.: From research to applications – Examples of operational ensemble post-processing in France using machine learning, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-7804, https://doi.org/10.5194/egusphere-egu2020-7804, 2020.

D2478 |
EGU2020-8724
Jon Olav Skøien, Peter Salamon, and Fredrik Wetterhall

Different statistical techniques are frequently employed to post-process the outcome of ensemble forecasting models. The main reason is to compensate for biases due to errors in model structure or initial conditions, and as a correction for under- or overdispersed ensembles.

Here we present analyses of the results from one these methods. We use the Ensemble Model Output Statistics method (EMOS; Gneiting et al., 2005) to post-process the ensemble output from a continental scale hydrological model - LISFLOOD (Van Der Knijff et al., 2010; De Roo et al., 2000). The model was calibrated at approximately 700 stations based on long term observations of runoff and meteorological variables. We use the same locations for calibration and verification of the 1-10 days forecasts of the model, based on ensemble and deterministic meteorological forecasts from ECMWF (51 ensemble members + 1 high-resolution), DWD (1 member) and COMSO-LEPS (16 ensemble members).

We calibrated the EMOS-parameters using the Continuous ranked probability score (CRPS). Whereas the post-processing improved the results for the first 1-2 days lead time, the improvement was less for increasing lead times of the verification period. As the post-processing is based on assumptions about the forecast errors, we will here present analyses of the ensemble output that can give some indications of what to expect from the post-processing.

 

Gneiting, T., Raftery, A. E., Westveld, A. H. and Goldman, T.: Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation, Mon. Weather Rev., 133(5), 1098–1118, doi:10.1175/MWR2904.1, 2005.

Van Der Knijff, J. M., Younis, J. and De Roo, A. P. J.: LISFLOOD: a GIS‐based distributed model for river basin scale water balance and flood simulation, Int. J. Geogr. Inf. Sci., 24(2), 189–212, doi:10.1080/13658810802549154, 2010.

De Roo, A. P. J., Wesseling, C. G. and Van Deursen, W. P. A.: Physically based river basin modelling within a GIS: The LISFLOOD model, in Hydrological Processes, vol. 14, pp. 1981–1992, John Wiley & Sons Ltd. [online] Available from: http://www.scopus.com/inward/record.url?eid=2-s2.0-0034254644&partnerID=tZOtx3y1, 2000.

 

How to cite: Skøien, J. O., Salamon, P., and Wetterhall, F.: Will post-processing always improve my forecasts?, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-8724, https://doi.org/10.5194/egusphere-egu2020-8724, 2020.

D2479 |
EGU2020-13825
André Düsterhus

Traditionally, verification of (ensemble) model predictions is done by comparing them to deterministic observations, e.g. with scores like the Continuous Ranked Probability Score (CRPS). While these approaches allow uncertain predictions basing on ensemble forecasts, it is open how to verify them against observations with non-parametric uncertainties.

This contribution focuses on statistically post-processed seasonal predictions of the Winter North Atlantic Oscillation (WNAO). The post-processing procedure creates in a first step for a dynamical ensemble prediction and for a statistical prediction basing on predictors two separate probability density functions (pdf). Afterwards these two distributions are combined to create a new statistical-dynamical prediction, which has been proven to be advantageous compared to the purely dynamical prediction. It will be demonstrated how this combination and with it the improvement of the prediction can be achieved before the focus will be set on the evaluation of those predictions at the hand of uncertain observations. Two new scores basing on the Earth Mover's Distance (EMD) and the Integrated Quadratic Distance (IQD) will be introduced and compared before it is shown how they can be used to effectively evaluate probabilistic predictions with uncertain observations. 

Furthermore, a common approach (e.g. for correlation measures) is to compare predictions with observations over a longer time period. In this contribution a paradigm shift away from this approach towards comparing predictions for each single time step (like years) will be presented. This view give new insights into the performance of the predictions and allows to come to new understandings of the reasons for advantages or disadvantages of specific predictions. 

How to cite: Düsterhus, A.: Verification of post-processed seasonal predictions, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-13825, https://doi.org/10.5194/egusphere-egu2020-13825, 2020.

D2480 |
EGU2020-17886
Stephan Hemri, Christoph Spirig, Jonas Bhend, Lionel Moret, and Mark Liniger

Over the last decades ensemble approaches have become state-of-the-art for the quantification of weather forecast uncertainty. Despite ongoing improvements, ensemble forecasts issued by numerical weather prediction models (NWPs) still tend to be biased and underdispersed. Statistical postprocessing has proven to be an appropriate tool to correct biases and underdispersion, and hence to improve forecast skill. Here we focus on multi-model postprocessing of cloud cover forecasts in Switzerland. In order to issue postprocessed forecasts at any point in space, ensemble model output statistics (EMOS) models are trained and verified against EUMETSAT CM SAF satellite data with a spatial resolution of around 2 km over Switzerland. Training with a minimal record length of the past 45 days of forecast and observation data already produced an EMOS model improving direct model output (DMO). Training on a 3 years record of the corresponding season further improved the performance. We evaluate how well postprocessing corrects the most severe forecast errors, like missing fog and low level stratus in winter. For such conditions, postprocessing of cloud cover benefits strongly from incorporating additional predictors into the postprocessing suite. A quasi-operational prototype has been set up and was used to explore meteogram-like visualizations of probabilistic cloud cover forecasts.

How to cite: Hemri, S., Spirig, C., Bhend, J., Moret, L., and Liniger, M.: Towards operational postprocessing of cloud cover at MeteoSwiss, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-17886, https://doi.org/10.5194/egusphere-egu2020-17886, 2020.

D2481 |
EGU2020-18784
Jose L. Casado-Rubio, Isabel Martínez-Marco, and Carlos Yagüe

Direct normal irradiance (DNI) forecasts from two ensemble models, the global ECMWF-ENS and the limited area multimodel gSREPS, have been calibrated using the quantile regression method, taking DNI as the only input parameter to better understand the inner workings of the method. Forecasts for the southern part of Spain, with lead times up to 72 hours for ECMWF-ENS and 24 hours for gSREPS over a two-year period (from June 2017 to May 2019), have been used.

This study has focused on two particular aspects of the postprocess:

  • The effect of quantile regression on the spread of the models. The results show that the spread of ECMWF-ENS greatly increases after the postprocess, which has a positive effect on the accuracy of the model, with an improvement of 20% in the continuous ranked probability score (CRPS) after the calibration. However, this increase is uniform over the whole period, affecting equally to situations with low or high spread, hence the postprocessed forecasts are not able to detect changes in predictability. On the other hand raw gSREPS forecasts behave better during episodes of both low or high predictability. The postprocess does not significantly change the spread and accuracy of gSREPS.
  • The influence of the training sample. It has been found that DNI is a variable which can experience periods of low variability, particularly in regions like southern Spain, where long spells of sunny days are common. This has a sizeable impact on the performance of the quantile regression on certain days. Two study cases will be shown to illustrate this problem. Two possible solutions are proposed: use longer training periods (not always possible) or place restrictions on the value of the regression coefficients.

How to cite: Casado-Rubio, J. L., Martínez-Marco, I., and Yagüe, C.: Calibration of direct normal irradiance (DNI) forecasts with quantile regression, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-18784, https://doi.org/10.5194/egusphere-egu2020-18784, 2020.

D2482 |
EGU2020-22298
Jan Rajczak, Regula Keller, Jonas Bhend, Christoph Spirig, Stephan Hemri, Lionel Moret, and Mark Liniger

MeteoSwiss is currently developing a post-processing suite for the territory of Switzerland. The system aims to provide optimized multi-variable (i.e. temperature, precipitation, wind and cloud cover), spatial and probabilistic predictions. The system will combine information in a seamless manner from the in-house short range and regional (COSMO-E/1) of 1 resp. 2 km resolution and the medium range ECMWF IFS NWP systems. At the example of probabilistic temperature forecasts, this contribution discusses recent advances and experiences at developing, applying and operationalizing non-homogenous Gaussian regression, also known as ensemble model output statistics (EMOS).

Over the complex terrain of Switzerland, postprocessing leads to a substantial improvement of temperature forecasts by up to 30% in terms of CRPS with respect to elevation-corrected direct model output (DMO) even by a basic EMOS only relying on DMO of temperature. Incorporating suitable predictors, such as the atmospheric boundary layer height, leads to a further gain in forecast quality. Results also show that combining high- (COSMO-E) and coarse-resolution (IFS) NWP output can not only provide a seamless medium-range forecast, but also further increase prediction skill during the time horizon when both models are available. Finally, we discuss first attempts to produce high-resolution spatial PP fields for arbitrary locations by exploiting a global EMOS framework with multiple static (e.g. geographic characteristics) and dynamic predictors derived from NWP data.

How to cite: Rajczak, J., Keller, R., Bhend, J., Spirig, C., Hemri, S., Moret, L., and Liniger, M.: Towards operational post-processing of probabilistic temperature forecasts at MeteoSwiss, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-22298, https://doi.org/10.5194/egusphere-egu2020-22298, 2020.

D2483 |
EGU2020-22422
| solicited
Sam Allen, Christopher Ferro, and Frank Kwasniok

A number of realizations of one or more numerical weather prediction (NWP) models, initialised at a variety of initial conditions, compose an ensemble forecast. These forecasts exhibit systematic errors and biases that can be corrected by statistical post-processing. Post-processing yields calibrated forecasts by analysing the statistical relationship between historical forecasts and their corresponding observations. This article aims to extend post processing methodology to incorporate atmospheric circulation. The circulation, or flow, is largely responsible for the weather that we experience and it is hypothesized here that relationships between the NWP model and the atmosphere depend upon the prevailing flow. Numerous studies have focussed on the tendency of this flow to reduce to a set of recognisable arrangements, known as regimes, which recur and persist at fixed geographical locations. This dynamical phenomenon allows the circulation to be categorized into a small number of regime states. In a highly idealized model of the atmosphere, the Lorenz ‘96 system, ensemble forecasts are subjected to well-known post-processing techniques conditional on the system's underlying regime. Two different variables, one of the state variables and one related to the energy of the system, are forecasted and considerable improvements in forecast skill upon standard post-processing are seen when the distribution of the predictand varies depending on the regime. Advantages of this approach and its inherent challenges are discussed, along with potential extensions for operational forecasters.

How to cite: Allen, S., Ferro, C., and Kwasniok, F.: Regime-dependent statistical post-processing of ensemble forecasts, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-22422, https://doi.org/10.5194/egusphere-egu2020-22422, 2020.