Workflow Modernization for Open and Scalable Access to Operational NWP Data

Operational Numerical Weather Prediction (NWP) workflows are increasingly challenged by rapidly growing data volumes, expanding product diversity, and the need for timely and scalable access to model data. At the same time, modern Earth system services are evolving toward open data policies that require not only standardized access to model output for internal and external users, but also flexible mechanisms to extract and process relevant information in a FAIR (Findable, Accessible, Interoperable, and Reusable) manner. In this context, MeteoSwiss, in collaboration with the European Centre for Medium-Range Weather Forecasts (ECMWF), is developing a modernized data workflow to improve access to NWP model data for internal and external downstream users. 

The redesigned workflow shifts from a product-centric dissemination model toward a scalable data-as-a-service approach. Rather than relying on the generation and distribution of numerous predefined products, recent ICON forecast output is organized in the Field Database (FDB) and exposed through Polytope, which provides semantic data access and feature extraction capabilities. The workflow automates the ingestion, indexing, access control, and on-demand extraction of forecast fields, and integrates these steps into existing HPC-based production workflows and downstream processing pipelines. By replacing file-based product generation with database-backed access, the workflow enables deterministic data extraction, explicit provenance tracking, and consistent versioning of datasets, so that identical data requests can be reproduced reliably across time and environments. We present recent developments in Earthkit and Polytope that, for the first time, enable such automated workflows on the icosahedral grids used by ICON. Standardized interfaces and modern processing tools from the Earthkit Python ecosystem enable downstream users and applications to retrieve and process tailored subsets of NWP data on demand. 

Our use of open-source, community-developed software (FDB, Polytope, Earthkit) as core workflow components illustrates how ECMWF technologies can be integrated into national weather service environments. Operational experience gained in this context contributes to improving the maturity and usability of these tools and supports their broader adoption by other ECMWF Member States, facilitating the transfer of FAIR, workflow-based data access concepts across the weather and climate community.