Evaluation of co-creation processes in climate services - Development of a formative evaluation scheme

 

Climate change increasingly affects all parts of society. Different economic sectors such as the agricultural sector have to adapt to climate change. More and more climate services are being developed in order to support this adaptation to climate change with accurate and suitable products. Good practises for the design of climate services include transdisciplinary approaches and co-creation of climate service products. The development of usable and useful climate service products and effective adaptation measures requires constant interactions between climate service providers and users of the products. To assess the effectiveness of these co-creation endeavours, continuous evaluation is crucial. At present, output and outcome assessments are conducted occasionally in this research field. However, these summative evaluations that are preformed ex-post do not help to adjust the ongoing process of co-creation. Therefore, the focus of the presented work is on formative evaluation of the co-creative development of science-based climate service products. A formative evaluation is done during the run-time of a project with the intention to reflect and readjust it. For this purpose, we analysed in detail the process of co-creation of climate service products in the knowledge transfer project ADAPTER (ADAPT tERrestrial systems, https://adapter-projekt.org/) and combine this analysis with a systematic literature review. In ADAPTER, simulation-based climate service products are developed together with key partners and practitioners from the agricultural sector, with the aim of supporting decision making in the context of climate change adaptation.

As a first step, main characteristics of the product development process were identified empirically and six sub-processes of product development were determined. Secondly , questions for a formative evaluation were assigned to the different steps and sub-processes. Thirdly, a literature review including fields other than climate services delivered additional qualitative aspects. As a result, a scheme of quality criteria and related assessment questions for the different sub-processes in climate service development was created, based on both empirical and theoretical work. Subsequently, this scheme needs validation and testing. The resulting formative evaluation scheme will be particularly helpful to reflect on and to improve the co-creation processes in climate services and beyond.