DAFNI: Building a VRE for National Infrastructure

The DAFNI Platform (Data and Analytics Facility for National Infrastructure) is a Virtual Research Environment which stores data and software models, provides an execution environment for those models, and supports data visualisation. Formally launched in July 2021, the DAFNI Platform was built to support national infrastructure research. 

Infrastructure systems supplying water and energy, transport networks, communication networks, and waste management provide the backbone of modern societies and play a key role in the development of nations and communities. The infrastructure research community is diverse, and collaboration between domains is complex, with different development teams using different data and programming standards.  Further, differences in data formats, in spatial and temporal resolution, and in data semantics make it complex to work together and combine models for integrated impact assessments. 

Infrastructure systems cannot, however, be considered in isolation. The interactions between them need to be considered to determine their most effective design and operation. Furthermore, the need for resilience and adaptation to climate change must be examined by all domains. 

To support these heterogeneous research communities working together, the DAFNI Platform was built with flexibility at its core. Containerisation, object stores and high-level metadata vocabularies are some of the key technical aspects to this flexibility, along with a domain-agnostic user interface. When uploading data to the platform, users may upload any file format which is then stored without transformation in an object store. Software models are containerised and uploaded to the DAFNI Platform as Docker images, where they can then be executed on DAFNI as a workflow. Using containerisation, it is possible to combine models together in sequence irrespective of the language that they were written in or the OS on which they were created, allowing researchers to continue using their established practices. 

The decisions to create services which are domain-agnostic have also necessitated certain trade-offs, however, which may not apply to more specialised platforms. For example, while a high-level metadata schema can be applied to e.g. rail timetables as well as it can to flood extent data, it does not support accepted standards or ontologies in either rail or flooding research. Object stores, Docker containers, and neutral user interfaces also come with their own challenges.  

Despite these challenges, however, the DAFNI Platform offers a unique capability and has successfully supported many projects using complex model and data interactions. One prominent example of this is the OpenCLIM project which used the DAFNI Platform to develop workflows linking different human and infrastructure systems to environmental data - such as linking urban development with climate-driven rainfall changes and flooding - and continues to showcase this research on DAFNI beyond project lifetime. 

This presentation will showcase the DAFNI Platform’s functionality, explain key design decisions, and illustrate its impact through examples of research enabled by the platform. We will also reflect on lessons learned in building a VRE for a multidisciplinary domain and discuss implications for future infrastructure research environments.