Enhancement of quality in climate services – development of a formative evaluation scheme for co-creation processes

National and international climate services support adaptation to climate change. Climate services should aim to deliver climate service products, which are useable, useful, accurate and suitable for users. Good practices for the design of climate services include transdisciplinary approaches and co-creation of climate service products.

The advancement of effective adaptation measures to climate change requires continuous interactions between climate service providers and users of the products. To assess the effectiveness of this co-creation endeavours evaluation is crucial. At present, output and outcome assessments are conducted occasionally in the climate services research field but they are performed ex-post and do not help to adjust the ongoing process of co-creation.

Thus, the focus of the presented work is on the formative evaluation of the co-creative development of science-based climate service products. It is done during the run-time of a project with the intention to reflect and readjust the processes. To find the related evaluation criteria, we analysed in detail the process of co-creation of climate service products in the knowledge transfer project ADAPTER (ADAPT tERrestrial systems, https://adapter-projekt.org/). In ADAPTER, simulation-based climate service products have been developed together with key partners and practitioners from the agricultural sector, to support decision-making in the context of climate change adaptation.

Starting with the empirical identification of the main characteristics of the product development process, six sub-processes were determined. Then, questions for a formative evaluation were assigned to the different steps and sub-processes. Finally, a literature review on quality criteria in transdisciplinary research delivered additional qualitative aspects and served as theoretical basis. As a result, a scheme of quality criteria and related assessment questions for the different sub-processes in climate service development was created, based on both empirical and theoretical work. Of course, this formative evaluation scheme needs validation and testing but, at least, will be particularly helpful to reflect on and to improve the co-creation processes in climate services and enhance their quality.