Publishing computational research – A review of infrastructures for reproducible and transparent scholarly communication

Many research papers include results based on data that is analyzed using a computational analysis implemented, e.g., in R. Publishing these materials is perceived as being good scientific practice and essential for the scientific progress. For these reasons, organizations that provide funding increasingly demand applicants to outline data and software management plans as part of their research proposals. Furthermore, the author guidelines for paper submissions more often include a section on data availability, and some reviewers reject submissions that do not contain the underlying materials without good reason [1]. This trend towards open and reproducible research puts some pressure on authors to make the source code and data used to produce the computational results in their scientific papers accessible. Despite these developments, publishing reproducible manuscripts is difficult and time-consuming. Moreover, simply providing access to code scripts and data files does not guarantee computational reproducibility [2]. Fortunately, several projects work on applications to assist authors in publishing executable analyses alongside papers considering the requirements of the aforementioned stakeholders. The chief contribution of this poster is a review of software solutions designed to solve the problem of publishing executable computational research results [3]. We compare the applications with respect to aspects that are relevant for the involved stakeholders, e.g., provided features and deployment options, and also critically discuss trends and limitations. This comparison can be used as a decision support by publishers who want to comply with reproducibility principles, editors and program committees who would like to add reproducibility requirements to the author guidelines, applicants of research proposals in the process of creating data and software management plans, and authors looking for ways to distribute their work in a verifiable and reusable manner. We also include properties related to preservation relevant for librarians dealing with long-term accessibility of research materials.

 

References:

1) Stark, P. B. (2018). Before reproducibility must come preproducibility. Nature, 557(7706), 613-614.

2) Konkol, M., Kray, C., & Pfeiffer, M. (2019). Computational reproducibility in geoscientific papers: Insights from a series of studies with geoscientists and a reproduction study. International Journal of Geographical Information Science, 33(2), 408-429.

3) Konkol, M., Nüst, D., & Goulier, L. (2020). Publishing computational research - A review of infrastructures for reproducible and transparent scholarly communication. arXiv preprint arXiv:2001.00484.