ECMWF's data archive and dissemination services migration to the Bologna Data Center.

CMWF is the European Centre for Medium-Range Weather Forecasts. We are both a research institute and a 24/7 operational service, producing global numerical weather predictions and other data for our Member and Co-operating States and the broader community. The Centre has one of the largest supercomputer facilities and meteorological data archives in the world.

 

ECMWF is about to migrate his 400+ PB of data to his new data centre in Bologna while continuing its operations. We will present and discuss challenges and opportunities that this migration offers in terms of evolution of operation practices.

The planning, the evolution, and the transition periods of the ECMWF Data Handling System migration to Bologna will be presented.

 

The migration must occur while preserving ECMWF’s product generation and archive services, ensuring appropriate levels of quality of service. The planning and testing of a continuity plan of operations for operational forecasts, member states time critical suites, Copernicus suites (ERA5, CAMS C3S seasonal and alike), and research suites will be presented. This continuity plan of operation relies on the full identification and traceability of the data flow involves during critical operations. Indeed, it is not economically viable to keep the 400 PB online during all the migration period.

 

A completely redesigned data services deployment and testing mechanism will be use in the Bologna Data Center. Automation will be paramount in this context as the need is to redeploy entirely and from scratch all our services. This journey will be presented, and challenges inherent to software defined infrastructure and services will be discussed.