Hydrological simulations with seamless scaling between Could and High Performance Computing environments on DestinE from the comfort of your own browser using eWaterCycle.

The goal of the eWaterCycle project is to facilitate hydrological modelling being done Findable, Accessible, Interoperable & Reproducible (FAIR).

High (hyper) resolution and / or large sample hydrological modelling, including those driven by Destination Earth (DestinE) Digital Twin (DT) inputs, often require HPC infrastructure for model runs. Designing such studies, however, benefit from users working on interactive Cloud Infrastructures. Migrating workflows from Cloud Infrastructure to HPC infrastructure requires deep knowledge of the systems in place, which typical (hydrological expert) users don’t have. A core design philosophy of the eWaterCycle platform is that domain (hydrology) users should not need to become computer science experts to carry out their hydrological research.

 

To address this, we have developed a workflow that seamlessly upscales any hydrological workflow designed on Cloud Infrastructure to a SLURM high performance compute cluster, with small changes compared to working from the cloud environment: setting up paths (e.g. scratch folders) and supplying key argument parameters (e.g. the specified region). Users are not required to have any prior knowledge of HPC systems beforehand.

 

This 'seamless' workflow can be run from any jupyterhub environment. We are using and are in development of integrating with the services that are part of DestinE.

As a ‘large sample hydrology’-example: we run a climate change impact on flood frequency analysis on each of the 6830 catchments in the entire . We facilitate using ERA5, ERA-Interim and CMIP6 data as well as the data provided by the Digital Twin (DT) as input to these model runs. In this presentation we will be sharing our results obtained using eWaterCycle with the DT data. This workflow will serve as an example of our seamless scaling between Cloud Infrastructure and HPC systems and provide lessons learned for others setting up similar services.