The researcher effect can outweigh the original effect in meta-analyses of global change experiments

During meta-analyses, a critical stage consists of the numerical extraction of the data from the original publications. While in the medical disciplines, there are often stanard operating procedures on how exactly to extract those numbers that are then expressed as effect sizes. Because in global change ecology, we mostly lack such rules on data extraction, we often rely on the judgement of the researcher. For example, 'micro-decisions' have to be made as to what time window is averaged, or whether species are pooled or not. In an effort to create the over-arching MESI database (see van Sundert et al. 2023 GCB), amalgamating four existing data-bases, we identified a substantial 'researcher effect', which occasionally outweighs the originally observed effect of global change on ecosystems. For instance, we identified a substantial discrepancy between what different meta-analyses found in regards to the effect of rising atmospheric CO2 and warming on below ground biomass, ranging from net negative to net positive effects. Importantly, the meta-analyses are largey based on the same original data. We discuss this issue, which likely exists in other disciplines and show possible ways forward.