Adequacy and Reliability of Earth System Models: Actionable Purposes, Model Inadequacy and Epistemic Risk

For scientific products to be considered actionable for stakeholder purposes they must meet certain epistemic and contextual conditions of adequacy. In evaluating the adequacy of a scientific product derived from Earth system models for actionable purposes—such as adaptation or resilience planning— there is a tendency to engage in the evaluation of raw model output and subject it to postprocessing to gain desired reliability and fitness. However, this reductive approach and focus on data ignores questions of whether the simulations, model configurations, and representational features of the models are themselves adequate and reliable for the actionable applications. This talk will lay out the reasons why we need to shift our practices to evaluating models and their products in a more holistic manner and provides insight into a framework for doing so. Scientific models—in this case Earth system models—are constructed with certain purposes and research questions in mind. These purposes, and more detailed research questions, engender representational values, which are reflections of what we want to know and why we want to know it. When model development is informed by these representational values underlying our questions and purposes, they are determinants of the decisions made during model construction about what we choose to represent and how we choose to represent it. The consequence is that the models constructed reflect these representational values and occupy a representational perspective, one that is fit for answering the questions and purposes that governed its development, but not those questions and applications that lie outside that perspective. To avoid increasing epistemic risk when using models for actionable purposes, which can result in downstream social harms, we need to assess the adequacy and reliability of our instruments and their products further upstream, in terms of consistency between the representational values that are embedded in the model in virtue of its development pathway and those that are implied in the actionable science questions the model could be applied to answer. More holistic, tailored assessments will allow us to avoid increases in epistemic risk due to how stakeholder representational values and conditions of adequacy can be inconsistent with those values being reflected in the representational content of the model being employed.