Flexible and scalable workflow framework HydroFlows for compound flood risk assessment and adaptation modelling

Flood risk assessments are increasingly guiding urban developments to safeguard against flooding. These assessments, consisting mainly of hazard and risk maps, make use of interconnected models consisting of a chain of climate, hydrological, hydraulic, and impact models, which are increasingly run interactively to support scenario modelling and decision-making in digital twins. To maintain interoperability, transparency, and reusability of this chain and the assessments themselves, using a workflow manager to manage the inter-model dependencies is a natural fit. However, composing and maintaining workflows is a non-trivial, time-consuming task, and they often have to be refactored for new workflow engines, or when changing compute environments, even if the workflow conceptually remains unchanged. These issues are particularly relevant in the development of digital twins for climate adaptation, where flood risk assessments serve as input to indicate high-risk areas. The complex model chain underpinning such digital twins can benefit greatly from transparent workflows that can be easily reused across different contexts.

To address these challenges, we developed the HydroFlows Python framework for composing and maintaining flood risk assessment workflows by leveraging common patterns identified across different workflows. The framework allows users to use one of the many steps available in the library or define workflow steps themselves and combine these into complete workflows which are validated on the fly. Available workflow steps include building, running, and postprocessing of models. Execution of the workflow is handled by one of the workflow managers to which our workflow description can be exported, such as Snakemake or tools with CWL support. This flexibility allows users to easily scale their workflows to different compute environments whenever the computational requirements demand so.

We demonstrate the flexibility of the HydroFlows framework by highlighting how it can be used to create complex workflows needed for digital twins supporting climate adaptation. HydroFlows not only enhances the flexibility and portability of the digital twin modelling workflows but also facilitates the integration of digital twin tooling and advanced computing and processing solutions to support interactive flood risk assessments in federated compute and data environments.