Evaluating the quality of model-based regional climate information: the case of the UK Climate Projections 2018

The kind of long-term regional climate information that is increasingly important for making adaptation decisions varies in temporal and spatial resolution, and this information is usually derived from Global Climate models (GCMs). However, information about future changes in regional climate also comes with high degrees of uncertainty–an important element of the information given the high decision stakes of climate change adaptation.

 

Given these considerations, Baldissera Pacchetti et al. (in press) have proposed a quality assessment framework for evaluating the quality of regional climate information that intends to inform decision making. Evaluating the quality of this information is particularly important for information that is passed on to decision makers in the form of climate services. The framework has five dimensions along which quality can be assessed: diversity, completeness, theory, adequacy for purpose and transparency.  

 

Here, we critically evaluate this framework by applying it to one example of climate information for adaptation: the UK Climate Projections of 2018 (UKCP18). There are two main motivations for the choice of UKCP18. First, this product embodies some of the main modeling strategies that drive the field of climate science today. For example, the land projections produced by UKCP18 provide probabilistic uncertainty assessments using multi-model and perturbed physics ensembles (MME and PPE), use locally developed GCMs and the models from the international Climate Model Intercomparison Project (CMIP), perform dynamical downscaling for producing information at the regional scale and further fine grain information with convection permitting models. Second, the earlier version of the UK Climate Projections (UKCP09) has received criticism from philosophers of science. The quality assessment framework proposed by Baldissera Pacchetti et al. partly aims to reveal whether the pitfalls identified by philosophers in UKCP09 persist in UKCP18.

 

We apply the quality assessment framework to four strands of the UKCP18 land projections and illustrate whether and to what extent each of these strands satisfies the quality dimensions of the framework. When appropriate, we show whether quality varies depending on the variable of interest within a particular strand or across strands. For example, the theory quality dimension highlights that epistemic quality along this dimension is better satisfied for estimates about variables that depend on thermodynamic principles (e.g. global average temperature) than fluid dynamical theory (e.g. precipitation) (see, e.g., Risbey and O’Kane 2011) independently of the strand under assessment. We conclude that for those dimensions that can be evaluated, UKCP18 is not sufficiently epistemically reliable to provide information of high quality for all of the products provided.