EGU22-15, updated on 25 Mar 2022
EGU General Assembly 2022
© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

Quality assessment of meta-analyses on soil organic carbon research  

Julia Fohrafellner1, Sophie Zechmeister-Boltenstern2, Rajasekaran Murugan2, and Elena Valkama3
Julia Fohrafellner et al.
  • 1BIOS Science Austria, Dietrichgasse 27, 1130 Vienna, Austria
  • 2Institute of Soil Research (IBF), Department of Forest- and Soil Sciences, University of Natural Resources and Life Sciences, Vienna (BOKU), Gregor-Mendel-Straße 33, 1180 Vienna, Austria
  • 3Natural Resources Institute Finland (Luke), Bioeconomy and environment, Sustainability Science and Indicators. Tietotie 4, 31600 Jokioinen, Finland

The number of meta-analyses published in the field of agriculture is continuously rising. As a consequence of this rising popularity, more and more publications refer to their synthesis work as a meta-analysis, despite applying less than rigorous methodologies. All this gives reason to assume that core criteria, necessary in producing meta-analyses, are not clear to many researchers. As a result, poor quality meta-analyses are published, which might report questionable conclusions and recommendations to policymakers and farmers. This study is therefore aiming to provide fellow soil and agricultural researchers with an easy-to-use set of criteria on how to produce high quality meta-analyses. Alongside, the incorporated scoring scheme supports researchers and policy makers in evaluating the quality of existing agricultural meta-analyses.

We analyzed 31 meta-analyses studying the effects of different management practices on SOC between the years 2005-2020. Moreover, the retrieved meta-analyses were structured according to eleven management categories which allowed us to analyze and assess the quality of the state-of-knowledge on these categories. We found that, although overall quality was rising, meta-analyses on SOC still do not reach sufficient quality and a maximum score may be reached only by the year 2032.

Especially for the reporting of literature search, application of standard metrics for effect size calculation, correct weighting, extraction of independent effect sizes and database presentation, major deficiencies were found. In some cases, the term “meta-analysis” is still falsely used to describe quantitative syntheses of any style. Only one out of 31 meta-analyses in the category “tillage” applied a rigorous meta-analytical methodology and received a high overall quality score.

We conclude that, in order for the scientific community to provide high quality synthesis work and to push forward the sustainable management of agricultural soils, we need to adapt rigorous methodologies of meta-analysis as quickly as possible.

How to cite: Fohrafellner, J., Zechmeister-Boltenstern, S., Murugan, R., and Valkama, E.: Quality assessment of meta-analyses on soil organic carbon research  , EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-15,, 2022.


Display file