The Risk of Negatively Biased and Overconfident Return Level Estimates: A Critique of the Metastatistical Approach to Extremes

Classical extreme value analysis (EVA) often give large uncertainties on estimated return levels due to the limited length of real-world hydrological time series. The metastatistical extreme value (MEV) approach (Marani and Ignaccolo 2015) aims to overcome these limitations by describing all data using a common distribution, treating extremes as large ordinary data values. The above authors perform Monte Carlo simulations with synthetic time series generated from a Weibull distribution and fit a Weibull distribution to each series, as prescribed in the MEV approach. These simulations show that the MEV give unbiased estimates with smaller confidence intervals, compared with the GEV and Gumbel methods from classical EVA.

However, the MEV method neglects that physical mechanisms producing extremes often differ from those for ordinary events. Therefore, the ordinary and extreme events should in general be described by a mixture distribution and this may influence the results of MEV. To test this, we replicated their work and added a variant using synthetic time series from a Weibull mixture distribution, formed by mixing the original Weibull distribution with a tiny fraction of another Weibull distribution with a longer tail. This mimics the shift in distribution between ordinary and extreme events. When applying the Weibull-based MEV to the Weibull mixture samples, the MEV method produced systematically biased estimates, which are outside the confidence intervals provided by MEV. In contrast, GEV produced unbiased estimates that are inside the confidence interval.

Finally, goodness-of-fit tests are not able to distinguish between time series distributed according to Weibull and Weibull mixture, and can therefore provide no guidance on when to use MEV. In summary, we find the MEV approach unreliable for real-world applications and strongly caution against using it.