Notebook-as-a-VRE (NaaVRE): collaborative virtual labs to build digital twins of ecosystems

Solving many environmental challenges requires connecting a variety of datasets with advanced statistical or AI models, and to access distributed computing resources to create workflows or digital twins. Digital twins are particularly challenging because they require the integration of many datasets and models, resulting in complex interactions. Furthermore, researchers rely on interdisciplinary collaboration to build workflows and digital twins. To that end, they need to discover, reuse, and sometimes modify existing assets. While this can be supported by Virtual Research Environments (VREs), most existing solutions are tailored to a specific domain or use case, making it difficult to integrate external resources or to support the complex model composition required by digital twins.

To address these limitations, we have developed Notebook-as-a-VRE (NaaVRE), a VRE solution built on top of JupyterLab. NaaVRE enables researchers to create computational blocks by containerizing the cells of notebooks, to organize them into workflows, and to manage the full experimental cycle, including data and workflow sharing. The tool includes features such as metadata-driven resource discovery, workflow automation, and compatibility with external repositories. Designed for cloud infrastructures, NaaVRE provides cost-efficient and scalable solutions to support digital twin development.

Using NaaVRE, we build customized virtual labs to address specific scientific problems. These gather models, data access tools, workflows, and documentation into a shared space, where a community of users can develop, share and reuse them. We present a framework to guide the development and operations of virtual labs throughout their entire lifecycle.

We showcase NaaVRE and by building customized virtual labs for scientific data processing workflows and prototype digital twins. Those virtual labs are managed by LifeWatch ERIC and the University of Amsterdam following the aforementioned development framework. They include characterizing ecosystem structures with LiDAR data, tracking bird migration using radar, mapping invasive species, deriving essential variables in the context of the ENVRI-Hub NEXT project, and developing digital twins of ecosystems in the context of the Dutch NWO LTER-LIFE project.