Direct and harmonised access to Essential Climate Variable related in-situ observation data from SeaDataNet

SeaDataNet is a pan-European infrastructure that manages and provides access to marine datasets collected by European organisations through research cruises and observational activities in coastal waters, regional seas, and the global ocean. It was founded by National Oceanographic Data Centres (NODCs) and major marine research institutes. The network has expanded through successive EU-funded RTD projects and by contributing to major European initiatives such as EMODnet, Copernicus Marine Service, ENVRI, and the European Open Science Cloud (EOSC).

SeaDataNet develops and promotes widely adopted standards, vocabularies, software tools, and services that support FAIR marine data management. Its core service, the CDI (Common Data Index), provides unified online discovery and access to in situ marine observation data managed by more than 115 data centres in 34 countries. The service currently offers access to over 3 million datasets from more than 1,000 European organisations, covering physical, chemical, biological, geological, and geophysical data from European waters and the global ocean. The use of standard metadata, formats, and controlled vocabularies ensures rich, highly FAIR datasets.

SeaDataNet also delivers core data services for EMODnet Chemistry, Bathymetry, and Physics, harmonising large volumes of marine data that support the production of thematic data products, including Essential Climate Variable (ECV) and Essential Ocean Variable (EOV) datasets.

Environmental science increasingly relies on large, heterogeneous, and rapidly growing data collections that must be efficiently accessed, subsetted, and harmonised for use in models, digital twins, AI workflows, and Virtual Research Environments (VREs). The fully open-source Beacon software, developed by MARIS https://beacon.maris.nl/, addresses these challenges by enabling cloud-native, high-performance data lakes that are fast to deploy and access. Beacon supports parameter harmonisation using metadata annotations based on NERC vocabularies, ECV vocabularies, and the I-ADOPT methodology adopted in ENVRI-HUB Next.

To improve the ease of access to subsets of the SeaDataNet CDI data collection, a Beacon instance containing all the open SeaDataNet data was set-up. This now allows users to obtain real-time access to data subsets in multiple data formats (NetCDF, Parquet, Zarr) and flexible querying from Jupyter Notebooks or a newly developed Beacon studio (user interface) for non-technical users. Within ENVRI-HUB Next, this SeaDataNet instance enables on-the-fly access to ECV-related subsets from millions of files via Jupyter Notebooks, ready for use in the Analytical Framework.

The presentation focuses on this use case, the technical solution, and its potential applicability for other Research Infrastructures supporting EOSC use cases.