+++ EMS Open Data Stipend by ECMWF -- apply until 31 Mar 2024, 23:59 CEST -- see last part of session description +++
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Open Data policies have become both popular and mandatory across Europe. While several countries and institutions have adopted already a wide-open data policy, the EU Open Data Directive [1] is changing the landscape even more in the coming years.
In meteorology and climate science, a variety of European and international organizations provide access to vast amounts of relatively heterogeneous climatological data. Open data is having a growing impact on science and services, including more and more critical applications by both public and private sectors, by met services, companies, research institutes, and open source developers.
To tame the variety and sheer amount of data, API and data format standardization plays a key role in the ecosystem. Especially, the Open Geospatial Consortium (OGC) Environmental Data Retrieval (EDR) API has begun to permeate through the domain.
Additionally, it is more and more common to exploit clouds to distribute and process Open Data. Initiatives like European Weather Cloud [2], WEkEO [3], and Open Data on AWS [4] are specially built to bring users to data and make processing large data sets possible.
The aim of the session is to bring together the enablers, providers, and current/future users of Open Data in meteorology and climate, to share their experiences and requirements.
We invite contributions on both technical and user-focused topics related to
- New Open Data sets including hosting Open Data on-premise and in the cloud
- Effects of and preparing for the new EU Open Data Directive
- Metadata management including FAIR principles [5]
- Use of clouds to distribute and process Open Data, including cloud-native data formats
- Tools and interfaces (APIs) for distributing, accessing, and utilizing Open Data
- Existing Open Data applications using weather or climate data
- New ideas about where and how Open Data can serve society
- Opportunities and challenges regarding Open Data, including data sources, data formats, legal issues ...
- Community building and collaboration around Open Data
- Whatever you feel is necessary to tell about Open Data!
-------------------------------------------------------------------------------------------------------------------
+++ EODS - EMS Open Data Stipend - apply until 31 Mar 2024, 23:59 CEST +++
-------------------------------------------------------------------------------------------------------------------
We offer the opportunity to attend our Open Data session at EMS2024 with financial support from ECMWF: "EODS" is short for "EMS participation stipend for the Open Data Session, funded by ECMWF". This funding opportunity is meant for open source developers and other people active in open data who do not have an affiliation that can finance them attending EMS2024. We will support up to three persons as presenting authors in the session, either online or onsite depending on the number of applicants and contributions. We reimburse the chosen candidates for the online or onsite registration fee [6] plus one abstract processing fee [7] in our session.
Please note that EODS cannot offer financial support for any other costs beside above-mentioned abstract processing fees and registration fees.
Independent of this Open Data session stipend, EMS offers a registration fee waiver [8].
How EODS works:
Do not submit your abstract to EMS yet. Mail to open-data-community-editor@lists.ecmwf.int, including
- your abstract
- a very short CV
- a description of the lack of affiliation to finance EMS abstract and participation fees
- whether you applied for a registration fee waver
The deadline for sending your mail to us is 31 March 2024, 23:59 CEST.
We will notify you until 11 April if you win an EODS and if yes, if it is for online or onsite participation. Sou you will have another week after our notification to submit your abstract to our session via the EMS conference website.
We will notify the EMS organizing team of the winners - so you will not receive invoices but your invoice for the APF will be paid directly by ECMWF.
Remember to
- submit your abstract before 18 April via EMS website [9]
- make use of the early bird registration [6] - for online or onsite attendance according to our notification
Note that EODS cannot cover travel costs. It supports the abstract processing fee for the Open Data session plus early registration for online or onsite participation.
If you have any questions, please send a mail to open-data-community-editor@lists.ecmwf.int.
[1] https://digital-strategy.ec.europa.eu/en/policies/psi-open-data
[2] https://europeanweather.cloud
[3] https://www.wekeo.eu
[4] https://aws.amazon.com/opendata
[5] https://www.go-fair.org/fair-principles/
[6] https://www.ems2024.eu/venue_and_attend/registration.html
[7] https://www.ems2024.eu/abstract_management/how_to_submit_an_abstract.html#APF
[8] https://www.ems2024.eu/about/support/registration_fee_waiver.html
[9] https://www.ems2024.eu/about/updates/call_for_abstracts.html
The UK's open data journey, ignited in 2010, saw the public sector lead the charge, epitomised by Transport for London's (TfL) 2017 initiative, yielding £130 million in annual savings. Third-party developers thrived, leveraging TfL's data for innovative solutions, reshaping London's urban landscape. The Department for Environment, Food & Rural Affairs (DEFRA)'s 2015 open data surge birthed diverse applications, from flood risk assessments to archaeological discoveries. However this ‘big bang’ of open data was often done haphazardly and as a box-ticking exercise, so open data leaders shifted the paradigm from 'open it and they will come' to 'publish with a purpose'. The public sector responded with more powerful open data services such as DEFRA's Earth Observation Data Service, that remain as open as ever, but provide targeted around specific use cases.
On the other side, the private sector has come later to the party, and initially with less consistency, with open data rationales ranging from regulatory mandates to collaborative initiatives. Open Banking, despite its misleading moniker, epitomises this synergy and tension, spurred by government intervention demanding open data and secure data sharing. The ecosystem, fostered by open standards and APIs, burgeons with diverse offerings, from financial management to debt advice. Learning from this, initiatives like OpenActive, Open Energy and Stream, in the water sector, employ incentives to cultivate similar data ecosystems organically. Spearheaded by organisations like the ODI and Icebreaker One, these platforms facilitate open data and secure data sharing while fostering societal benefits.
Recognizing the wealth of data held by private entities, visionary companies embrace collaboration as the cornerstone of innovation. Microsoft's Industry Data for Society Partnership, in partnership with ODI and GovLab, spearheads this movement, bridging the data divide globally. Through initiatives like the Planetary Computer, Microsoft champions open data infrastructure, fostering collaborations to tackle pressing issues like climate change and education. Guided by principles of openness, usability, and empowerment, these endeavours pave the way for responsible data sharing, propelling society towards a more data-inclusive future.
Currently, we are beginning to recognise the best stewards of data are often neither government nor private sector, but independent vehicles set up to steward data on behalf of society for a specific purpose related to social impact. These organisations, data institutions, can publish open data, share sensitive data responsibly, and maintain key infrastructure for open data such as standards, identifiers and policies. Although nascent in their conceptual understanding, many already exist in the environment and climate space, providing access to data about climatic trends, forest cover, extractive industries and more. These organisations, operating in more networked and decentralised ways may be the future of the open data, but their impact is only just beginning to be understood.
How to cite: D'Addario, J. and Achenbach, K.: The State of UK Open Data 2024: From big bang open data to responsibly stewarding data with a purpose, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-413, https://doi.org/10.5194/ems2024-413, 2024.
ECMWF is a leading provider of weather information, and its products and weather data make contributions to a huge range of activities in service provision, research and for the protection of life and property via National Weather Services and humanitarian agencies.
Since 2020, ECMWF has been implementing an open data policy, recognising that open data under Creative Commons licensing is an essential tool in contributing to the development of new meteorological methods and is encouraging innovation and progress in the weather and climate domains.
ECMWF plans to achieve this goal by 2026. A number of steps have been taken to increase the open data available to users and in the last year ECMWF has built on this further. The data currently released as open data are a subset of ECMWF Real-time catalogue at 0.25 degree resolution, from both ECMWF Integrated Forecasting System (IFS) and Artificial Intelligence/Integrated Forecasting System (AIFS).
The usage statistics on ECMWF open data for 2023 show a significant increase of user adoption with respect to the previous year, indicating the users are engaging with ECMWF open data offerings.
ECMWF Open Data are provided through different mechanisms, all of which have seen a notable growth in usage in 2023:
- Open Charts Service, which provides chart visualisations of ECMWF products. Some of the increased uptake can be attributed to the introduction of charts based on Machine learning products in mid-2023.
- The Open Data Portal (FTP service) is available since January 2022 and shows a continuing increase in numbers of requests. ECMWF predicts even higher usage with the addition of new Open Data of IFS products at higher resolution and additional parameters, plus AIFS products (machine learning-based) in early 2024.
- WMO Essential and WMO Additional datasets (also provided via FTP service) are still increasingly popular. ECMWF will, in the future, optimise this provision, possibly merging it with the open data offering.
- Since January 2022, ECMWF Open Data is also redistributed by 3rd ECMWF has partnered with different cloud providers as part of their public dataset program (e.g. Amazon, Google and Microsoft), but the data has also started being provided by smaller independent users (e.g. Open-meteo.de) and in research archives (e.g. NCAR). The benefit is not only a good user uptake but also that data can be shared with a larger user community with no additional load on ECMWF systems.
ECMWF is committed to moving forward to achieve the goal of an open data policy over the next few years. Creative Commons licences will be applied to further datasets, while reducing and eventually removing the cost of data. The main objective of these changes is to continue to encourage collaboration, innovation and progress within the meteorological and climate science communities.
How to cite: Parodi, I., Coughlan, R., Kuilman, M., Pidduck, E., Bennett, V., Yang, X., and Modigliani, U.: Uptake of the ECMWF Open Data Programme, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-749, https://doi.org/10.5194/ems2024-749, 2024.
ECMWF are moving towards an extensive open data policy, providing data to a broader user base extending beyond operational forecasters in Member and Co-operating states and commercial customers. Beginning in 2020, the first phase saw the opening of hundreds of web forecast charts (www.charts.ecmwf.int) and made archived data available under a Creative Commons (CC BY 4.0) open licence. This momentum continued in January 2022 with the introduction of an open subset of real-time medium-range forecast data, with ongoing updates incorporating new parameters and datasets. Notably, the latest update in February 2024 included increasing the resolution from 0.4° to 0.25° and included the new Artificial Intelligence/Integrated Forecasting System (AIFS) model data.
This phased move towards free and open data aims to support creativity, innovation and reproducibility in scientific research and weather applications. However, this can not be achieved by only opening the real time data. The users need to be able to find and easily use the data and integrate it into their own research work or application workflows.
To address this, additional efforts are underway to improve the data's FAIR (Findable, Accessible, Interoperable and Reusable) attributes. Key developments include the creation of open source Python libraries for data processing and visualisation, alongside the introduction of a set of Jupyter notebooks, each of which is reproducing one open weather forecast chart - from the downloading the data to the visualisation.
However, the tools and data constantly change, and keeping up with these changes presents a significant challenge if not designed with maintenance in mind.
This talk will provide an overview of the open forecast web charts and the use of Jupyter notebooks for their reproduction, followed by an exploration of the maintenance challenges and future plans.
How to cite: Vuckovic, M.: Making ECMWF open weather forecast data more accessible to users: maintenance challenges, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-815, https://doi.org/10.5194/ems2024-815, 2024.
The WegenerNet climate station network Feldbach Region has provided measurements of hydrometeorological near-surface and surface variables with very high spatial and temporal resolution since 2007. It was specifically built as a long-term weather and climate monitoring facility that provides an open-ended data record to help fill the gap between short-term, high-resolution observations and long-term observations at larger scales. In 2020, this ground station network was complemented by the WegenerNet 3D Open-Air Laboratory for Climate Change Research, which has added atmospheric sounding capabilities through a polarimetric X-band Doppler weather radar, a microwave radiometer for vertical profiling of temperature, humidity, and cloud liquid water, an infrared cloud structure radiometer, and a water-vapor-mapping GNSS station network. Together, these facilities provide a unique setup for studying extreme meteorological events in a changing climate.
To maximize the impact of these data an open data store and open science tools, based on widely adopted and recognized (metadata) standards, are in the process of being implemented as part of the project WEGN2OPEN. Its focus lies on the provision of innovative high-resolution data cubes with a high level of interoperability to allow for an easy integration into existing applications and workflows.
The aim of this contribution is to showcase the current state of the WEGN2OPEN data store, along with planned advancements and features. This includes an overview of the currently available APIs through which datasets can be discovered and accessed, the web application for browsing, viewing, and subsetting data, and an outlook to a future extension with cloud-based analysis and processing tools.
How to cite: Kvas, A., Fuchsberger, J., Gorfer, M., Kirchengast, G., Scheidl, D., Bichler, C., Galovic, R., and Foelsche, U.: WEGN2OPEN - The WegenerNet Open Data and Science Laboratory Region Southeastern Austria, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-958, https://doi.org/10.5194/ems2024-958, 2024.
The modernized Copernicus Climate and Atmosphere Data Store infrastructure – released in 2024 – uses state-of-the-art technology to provide a more scalable and open access to data and services with the adoption of the FAIR principles and Open Data at its core. The infrastructure, named the Common Data Stores – Engine (CDS-E), exposes to the public two different frontends, the Climate Data Store (CDS) in support of the Copernicus Climate Change Service (C3S) and the Atmosphere Data Store (ADS) for the Copernicus Atmosphere Monitoring Service (CAMS) whilst sharing all the backend components. Combined, the CDS and ADS provide service to more than 300k registered users, delivering over 180 TBs of data on daily basis in the form of more than 700k processed requests. It also seeks to strengthen interdependencies with other European platforms such as the WEKEO Data and Information Access Platform (DIAS), Climate-Adapt, ClimSA, and facilitates interactions with different platforms in the future E.U. Green Deal Data Space, thereby fostering a broader engagement with the user community.
The CDS and ADS allow free and open access to the wide range of climate and atmospheric monitoring datasets through both a web-based and programmatic interface. The modernized infrastructure makes use of modern cloud technology and an open framework that bring new components to the CDS and ADS. These include a variety of adaptors, modern standards and protocols, such as OGC compliant APIs, STAC metadata standards for programmatic interrogation of geospatial information, Serverless Analysis-Ready Cloud Optimized (ARCO) Services and Standard WMS/WMTS for improved integration, access, and visualisation. This allows the CDS and ADSs to support a seamless experience on the user's journey to explore, discover, retrieve and use the available resources whilst remaining scalable and versatile.
At the core of its versatility is the implementation of FAIR principles (Findable, Accessible, Interoperable, Reusable). The adoption to the FAIR principles has in fact followed a gradual and systematic approach that has kept the platform and its content under continuous evolution and improvement. Here we describe how the adoption of the FAIR principles and an Open Data philosophy was implemented and outline the practical benefits of this approach to the user community.
How to cite: Zanacchi, M., Alos, A., Fuentes, M., and Damasio-Da-Costa, E.: The modernized Climate and Atmosphere Data Store infrastructure – a practical implementation of FAIR and Open Data, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-1089, https://doi.org/10.5194/ems2024-1089, 2024.
Over the past year, we have conducted a feasibility study and initiated a pre-project phase for the Open Earth Platform Initiative, aimed at harnessing open data and technology to drive climate adaptations. This initiative seeks to stimulate local innovation underpinned by global investment, thereby facilitating projects that are not only of higher quality and lower risk but also have a shorter development time from idea to product.
A key challenge identified is the scarcity of platforms that truly embrace an open approach. Vendor lock-in hinders the accessibility of open data, which often leads to both duplication of efforts and a prolonged process from data acquisition to product development. To combat these issues, our platform adheres to the FAIR principles, promoting findability, accessibility, interoperability, and reusability of data.
In the realm of open-source software, we emphasize the benefits of reuse and active involvement. While addressing the common concerns of security and vulnerabilities, we advocate for the synergy between open source software and open data, which enhances the overall effectiveness and innovation of the platform.
OpenEPI is designed with software developers as the primary end-users. A central feature is our developer portal, which is pivotal for ensuring developer engagement and productivity. The architecture of the platform champions vendor independence and utilizes a multicloud strategy to optimize performance and scalability. We rely on best-of-breed technologies, predominantly Python and its modern libraries and frameworks, alongside robust containerization practices.
In this talk at EMS2024, we will delve into the findings from our pre-project phase and provide updates on the current status, discussing how our strategic frameworks and practical implementations are being tailored to collectively foster the development of sustainable and impactful climate adaptation solutions.
How to cite: Stigen, K.: Open Earth Platform Initiative, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-1047, https://doi.org/10.5194/ems2024-1047, 2024.
When data is made accessible in the cloud, you can shorten the time to insights and help build impactful offerings for climate modelers and stakeholders. When data is shared on the Registry of Open Data on AWS, anyone can analyze it and build services on top of it using a broad range of compute and data analytics products, including Amazon EC2, Amazon Athena, AWS Lambda, and Amazon EMR. Sharing data in the cloud lets data users spend more time on data analysis rather than data acquisition. Just putting data into the cloud isn’t enough. In this session, the Amazon Web Services (AWS) Open Data team will discuss different options for creating cloud-optimized versions of data, that can be more efficiently utilized in the cloud. We will discuss the various options and trade-offs, such as when to create a full cloud optimized copy versus when to create a subset for your user community. We discuss how to match up the different options based on different levels of traffic, newness of data, and typical types of processing of the data. We will show how to use high quality climate and environmental datasets with the Registry of Open Data on AWS when they are cloud optimized and when they are not, and discuss the time difference for the data consumer between each option. Additionally, we will also show how to harness these data using low- and no-code machine learning tools such as SageMaker and SageMaker Studio Lab, using cloud-optimized capabilities such as subsetting, and how these capabilities can speed up the time from data to decision.
How to cite: Stoner, C.: Open Data for Impact: how to use the Open Data on Amazon Web Services to advance analysis, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-1120, https://doi.org/10.5194/ems2024-1120, 2024.
The RODEO project responds to the requirements of the EU Directive (2019/1024) on Open Data and its Implementing Regulation in order to boost the re-use and combination of open public data across the EU. The Implementing Regulation defines High Value Datasets (HVD) for six thematic categories of public data of which weather observation data, climate data, warnings, weather radar data and Numerical Weather Prediction (NWP) data are defined as meteorological High Value Datasets. These HVDs shall be shared free of charge, under the conditions of the Creative Commons BY 4.0 licence or a less restrictive open licence, and will be openly accessible via Application Programming Interfaces (APIs), machine-readable and bulk downloadable.
The three-year RODEO project is a joint effort by 11 European meteorological institutions, the European Centre for Medium-Range Weather Forecasts (ECMWF) and the network of 31 European National Meteorological and Hydrological Services, EUMETNET. The project strengthens the capacity of the European meteorological data providers by
- Developing a user interface and a data catalogue for making data discoverable;
- Developing APIs, using open licences, for accessing weather observation data, climate data, weather radar data, warnings, and AI datasets;
- Engaging with the data owners and user communities; and
- Supporting the deployment of national data portals and APIs.
- Implementing a comprehensive user engagement strategy involving
- A Project External Advisory Board, composed of public and private sector partners
- Frequent communications updates through several channels
The project is now in its second year. It has moved on from its design phase into developing and testing some of the APIs that are going to enable a large increase in the open availability of near real time European meteorological data. This will bring large benefits across society as a whole from the public through to decision makers. In particular, small and medium-sized enterprises (SMEs) will benefit from the greater data availability by creating new digital products and services and eventually attract new investors.
How to cite: McCairns, W.: RODEO Project – bringing meteorological data open for all users, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-1071, https://doi.org/10.5194/ems2024-1071, 2024.
Providing access to observational climate data and its metadata has been important for climatology. Historically, climate data has been made available by some national data platforms or through the European Climate Assessment and Database (ECA&D). Users often had to pay for data access and output was provided in different formats. The newly in force EU open Data Directive 2019/1024 requires National Meteorological and Hydrological Services (NMHSs) to make their High Value Datasets, including meteorological datasets, available through machine readable application programming interfaces (APIs). RODEO work package 5 (WP5) aims to unite ten NMHSs that are developing APIs for their climate databases. Afterwards, an API aggregator service unifies the individual APIs, so that the clients can use all APIs with only one request. In this presentation we will report on progress made in RODEO WP5.
WP5 worked on uniforming climate datasets, including its metadata. A group of experts decided on the choice of variable to include, granularity of daily and hourly observations, and preferred controlled vocabularies. Metadata guidance from WMO, Copernicus and ESIP are incorporated to adhere to the FAIR principles.
The API recommendations for NMHS data providers will follow the Environmental Data Retrieval (EDR) API specification. Using an EDR API it is possible to perform geospatial and temporal filterable queries of climate data. The EDR specification, published by Open Geospatial Consortium (OGC) in 2016, has, over recent years, gained traction as an ideal method to distribute meteorological data over the internet.
The OGC has adopted CoverageJSON as data format standard alongside the EDR API. This format will be the preferred output format for an NMHS climate data API. The EDR aggregator service will facilitate the unification of individual NMHS APIs to one central EDR API. The aggregator service can add caching, output format conversion from CoverageJSON to NetCDF or CSV and support queries across collections.
To aid in the development of NMHS climate data EDR APIs, a workshop “Build an EDR API in a day” is being prepared to run in June 2024. A spin-off of this event is being planned in Spring 2025 for non-RODEO partners.
How to cite: van Schayck, J. P., van der Schee, M., Phaf, L., and Kruithof, G.: RODEO WP5: Towards unified and uniform access to European climate data using the OGC EDR API , EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-701, https://doi.org/10.5194/ems2024-701, 2024.
Recently there has been many initiatives in the meteorological community and outside on implementing OGC API interfaces, especially OGC API EDR. Based on these efforts it seems that OGC API EDR will be a foundation technology in many initiatives and domains including WMO WIS2 (1), EUMETNET FEMDI (2) (aiming to provide high-value datasets required to the EU open data directive) and ICAO SWIM (3) services in the aviation domain. This presentation will touch on how the challenges are adressed in context of the RODEO project between participants in the project. In the project it was noticed that the implementations of different insitutes complied with the standard but were in fact still not compatible as the output was different. This led to the need and discussion on how we would achieve more interoperability between the actual output content provided by the API's. It was clear that some kind of agreement was needed between the partners in the project. Work on this challenge has also been started in the OGC context as well to add support for so called profiles. Challenges, current status of this work and lessons learned during the development will be presented.
(1) WMO Information System 2.0: framework for WMO data sharing
(2) EUMETNET FEMDI: Federated European Meteorological Data Infrastructure. Unified access to European weather data.
(3) ICAO SWIM: System Wide Information Management concept consists of standards, infrastructure and governance enabling the management of ATM related information and its exchange between qualified parties via interoperable services (ICAO Doc.10039). This includes weather information.
How to cite: Visa, M. and Rauhala, M.: Lessons learned on implemenation of the OGC EDR API, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-1024, https://doi.org/10.5194/ems2024-1024, 2024.
The Nicolaus Copernicus Centre for Climate Change Research (CCCR) was established in 2019, as a university interdisciplinary unit. It currently consists of fifteen researchers from four faculties representing five separate scientific disciplines (Mathematics and Computer Science; Biological and Veterinary Sciences; Historical Sciences; and Earth Sciences and Spatial Management). Researchers from Poland and abroad are kindly invited to work at or with the CCCR.
The main goal of the CCCR is to research climate changes over the last millennium in Poland, Central Europe and polar areas (mainly the Arctic). This goal can be achieved primarily by collecting historical meteorological data in archives and scientific libraries around the world. Our research is part of a worldwide initiative (Brönnimann et al. 2019) to inventory and rescue meteorological data for historical times.
The CCCR repository has so far included historical data gathered during three scientific projects, i.e. CEArc, ExtremeWeather, and MORCLIM for Poland and Arctic areas from the Middle Ages (as indices based on descriptive sources) until Early Twentieth Century Warming (1920-1950). Here we present the method of data collection, their digitization, quality control (QC), post-processing and the structure of our repository according to FAIR (Findability, Accessibility, Interoperability, and Reuse) principles.
The work was supported by the National Science Centre, Poland, projects No. 2020/37/B/ST10/00710 and No. 2020/39/B/ST10/00653
References:
Brönnimann S., Allan R., Ashcroft L., Baer S., Barriendos M., Brázdil R., Brugnara Y., Brunet M., Brunetti M., Chimani B., Cornes R., Domínguez-Castro F., Filipiak J., Founda D., Gergis J., Grab S., Hannak L., García Herrera R., Huhtamaa H., Jacobsen K. S., Jones P., Jourdain S., Kiss A., Lin K. E., Lorrey A., Lundstad E., Luterbacher J., Moberg A., Mauelshagen F., Maugeri M., Maughan N., Neukom R., Nicholson S., Noone S., Nordli Ø., Ólafsdóttir K. B., Pearce P. R., Pfister L., Pribyl K., Przybylak R., Pudmenzky C., Rasol D., Reichenbach D., Řezníčková L., Rodrigo F. S., Rohde R., Rohr C., Skrynyk O., Slonosky V., Thorne P., Valente M. A., Vaquero J. M. Westcottt N. E., Williamson F., Wyszyński P. 2019: Unlocking pre-1850 instrumental meteorological records: A global inventory. Bulletin of the American Meteorological Society 100: ES389-ES413. doi: https://doi.org/10.1175/BAMS-D-19-0040.1
How to cite: Wyszyński, P. and Przybylak, R.: Historical Meteorological Open Data Repository at Nicolaus Copernicus University Centre for Climate Change Research, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-299, https://doi.org/10.5194/ems2024-299, 2024.
Recent developments in open data policies of meteorological agencies have much expanded the set of up-to-date forecast, reanalysis, and observational data that is publicly available to meteorological research and education. To make use of this open data, we have developed a set of 3-D and AI-based visualization products that extract and display meteorological information in novel ways.
In this presentation, we present visualization products derived from publicly available data from operational agencies including the German Weather Service (DWD) and the European Centre for Medium-Range Weather Forecasts (ECMWF). All our visualizations are created using open-source software, mostly using the interactive 3-D visualization tool “Met.3D”. Met.3D has primarily been developed for rapid exploration of gridded atmospheric data by interactive means, and has recently been extended with capabilities for batch-creation of visualizations and animations. Met.3D supports a variety of visual displays from traditional 2-D maps to novel 3-D visualizations of, for example, clouds, jet streams, and weather fronts. In recent work, we also investigated explainable-AI-based feature identification algorithms to detect meteorological structures including fronts and tropical cyclones.
We provide comprehensive documentation and ensure straightforward installation processes for easy adoption and use of the presented visualization products by the community, e.g., for use in weather forecasting courses at universities. In addition, we are currently developing a near-real-time website that comprehensively showcases our visualizations using current forecast and observation data. For reproducibility and further interactive visual analysis of the data, the website provides scripts and configurations that enable users to replicate the visualizations using a local Met.3D installation, thus providing additional benefit to users.
How to cite: Fischer, C., Vogt, T., Beckert, A., Fuchs, S., Radke, T., and Rautenhaus, M.: Novel 3-D and AI-based weather forecast products based on open data, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-949, https://doi.org/10.5194/ems2024-949, 2024.
EU regulations declared Meteorological datasets as High-Value Datasets (HVD). To comply with this, as Hungary's national meteorological service provider, we publish our data on a public portal and make it freely available since January 1, 2021. On our website, odp.met.hu, model forecasts, weather radar data, current measurements and observations, and climate data are downloadable for everyone. The data usage has increased significantly with a growing number of visitors, reaching almost 58 000 in the past three years. The total data downloaded in 2023 reached 17 000 GB. CC-BY-SA licence applies to all our data. Furthermore, HungaroMet participates in the EU-funded RODEO project, so in the foreseeable future, our services will improve by API functionalities. So it is of utmost importance to raise awareness of our openly available data and services, as well as to follow uptake fo this data by our user community.
Our openly available climate database (HuClim) is a key component which is widely used by researchers for tracking climate change in Hungary. For instance, it is a key component of the Hungarian part of the FORESEE database (Open Database For ClimatE Change-Related Impact Studies in Central Europe, FORESEE-HUN v1.0) as the observation-based data (1971–2021).
HuClim provides researchers with an accurate, measurement-based and quality-controlled dataset. Our data is utilised in comprehensive research topics, including land, biodiversity and urban planning. These studies produce a plethora of articles related to climate change effects and adaptations in Hungary, which are published for the nonprofessional public to help them understand our future. A few highlights from those include extreme wind storms related to a severe motorway accident, the increasing number of life-threatening, persistent heatwaves and the changing vegetation as a result of the drying climate in the country. Some of these applications and deducted results will be presented on the poster at the EMS conference.
How to cite: Németh, C. and Lábó-Szappanos, E.: Open Data policy at HungaroMet and its benefits for climate research, EMS Annual Meeting 2024, Barcelona, Spain, 1–6 Sep 2024, EMS2024-1149, https://doi.org/10.5194/ems2024-1149, 2024.
