GRAPEVINE project: Operational weather forecasting in HPC managed by an orchestrator in Kubernetes cluster.

To promote cloud and HPC computing, GRAPEVINE* project objectives include using these tools along with open data sources to provide a reusable IT service. In this service a predictive model based on Machine learning (ML) techniques is created with the aim of preventing and controlling grape vine diseases in the wine cultivation sector. Aside from the predictive ML, meteorological forecasts are crucial input to train the ML models and on a second step to be used as input for the operational prediction of grapevine diseases. To this end, the Weather and Research Forecasting model (WRF) has been deployed in CESGA’s HPC infrastructure to produce medium-range and sub-seasonal forecasts for the targeted pilot areas (Greece and Spain). The data assimilation component of WRF – WRFDA-   has been also introduced for improving the initial conditions of the WRF model by blending in observations from weather stations and satellite precipitation products (Integrated Multi-satellitE Retrieval for GPM – [IMERG]). The operational production of the aforementioned forecasts is achieved by the cloudify orchestrator on a Kubernetes cluster. The connection between the Kubernetes cluster and the HPC infrastructure, where the model resides, is achieved with the croupier plugin of cloudify. Blueprints that encapsule the workflows of the meteorological model and its dependencies were created. The instances of the blueprints (deployments) were created automatically to produce operationally weather forecasts and they were made available to the ML models via a thredds server.  Valuable lessons were learned with regards the automation of the process and the coupling  with the HPC in terms of reservations and operational production.  

*hiGh performAnce comPuting seRvices for preVention and coNtrol of pEsts in fruit crops