Are return period estimates from observational records reliable for low-likelihood heatwave events? A systematic evaluation

Primer: The record-shattering Pacific Northwest heatwave in late June 2021 challenged a key element of extreme event attribution, namely, the statistical evaluation of the event likelihood given historical records up to the event. The respective model, a non-stationary generalised extreme value (GEV) distribution depending on a global mean temperature covariate, suggested an infinite return period, or zero probability of reaching the event intensity in the year in which it was observed, based on the historical record. The apparent shortcoming of the method triggered a widespread debate about the general suitability of this statistical approach and its ability to provide informative insight in the context of extreme event attribution.

Research objective: The aim of this study is to first evaluate the quality of return period estimates for very rare heatwave events to determine whether or not the method can reliably characterise the event likelihood of rare extremes. We then assess the contributions of different factors to systematic deviations in tail estimates (such as high quantiles or return periods) relevant for rare event attribution statements. We consider both aspects associated with the statistical method, as well as such related to the attribution procedure.

Data & Methods: A robust evaluation of tail estimates requires vast amounts of homogeneous data. Our analysis is based on two transient historical and future (RCP8.5 and SSP3.7) initial condition large ensembles (84 and 100 members) and an extensive bootstrap dataset of extreme values simulated from parametric GEV distributions.

Results: We demonstrate that also in climate model experiments, events analogous to the 2021 heatwave are simulated, which, assessed with data up to the event, would have deemed to have zero occurrence probability. Thus, also within the climate model context, we find that the non-stationary GEV approach yields substantially biased exceedance probability estimates for low-likelihood events, thereby overestimating the respective return period or underestimating the likelihood of occurrence if the GEV distribution is based on a relatively short “historical” record. This systematic has become particularly pronounced in recent extreme events due to the emergence of a distinct climate change signal and high rate of warming.

Especially maximum likelihood estimates of the non-stationary GEV distribution are prone to systematically underestimate the shape parameter, and in consequence overestimate the return periods. We demonstrate that the bias arises because the GEV fit is restricted to rather short time series, and it is partially alleviated if a Bayesian estimation approach is used. Furthermore, widely used symmetric, so-called Wald-type maximum likelihood confidence intervals are found to be a rather inadequate and misleading measure of the estimation uncertainty in GEV-parameters and tail quantities like return levels. For these reasons, Wald-type confidence intervals should thus not be used for model evaluation purposes in extreme event attribution studies.