Co-evaluating urban climate services: perspectives from climate scientists, decision makers and boundary agents on what makes “good” services

Climate services seek to provide information that enables climate-informed decision making by non-climate scientists. Often, especially where climate services are co-produced, boundary agents (typically social scientists) act between these groups to facilitate the seamless flow of knowledge in both directions and create climate services that are scientifically accurate and apt for building climate resilience. Or at least that is the idealised aspiration. 

In practice, developing climate services that are both useful and used involves contending with a wide range of factors beyond the project boundaries, ranging from the current limitations of climate science to societal power (im)balances and to the fitness of purpose of any service to a decision context. Different actors involved in developing and using climate services view them in different ways and hold different preferences on what constitutes a successful climate service. Thus, creating criteria to evaluate a climate service has an inherent subjectivity and designing a holistic evaluation framework requires drawing out these perspectives and preferences from decision-makers, climate scientists and boundary agents, and then bringing them together. 

Impetus4Change (I4C, https://impetus4change.eu/) is a Horizon Europe project joining 18 institutions from 8 countries that aims to improve the quality and usability of near-term climate information in cities and regions. Throughout the entirety of the project we are simultaneously co-producing climate services in four Demonstrator cities: Barcelona, Bergen, Paris, and Prague. This involves three stages: co-exploring the problems, solutions and realities that decision makers face; co-designing mock-ups of climate services and then co-developing these through Adaptalabs (highly interactive, transdisciplinary hackathons). The entire process is co-evaluated to capture lessons learned and combine these with detailed analysis of climate adaptation knowledge networks to explore the services’ replicability.

This presentation will cover the steps taken to generate tailored frameworks for evaluating urban climate services, including the generation of ideas from 60 participants of the first Adaptalab, the synthesis of pillars of the framework, and the tailoring of these pillars to each of the four Demonstrator cities. Using the Barcelona case study as an example, we show that actor perspectives on what is important vary not just in terms of what to assess, but also when. We conclude with examples of how we might evaluate different aspects of the co-production process, its outputs and its outcomes and our experiences operationalising the framework.