EGU2020-21549
https://doi.org/10.5194/egusphere-egu2020-21549
EGU General Assembly 2020
© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

NetCDF: Performance and Storage Optimization of Meteorological Data

Valentín Kivachuk Burdá1 and Michaël Zamo2
Valentín Kivachuk Burdá and Michaël Zamo
  • 1Institut de Recherche Technologique Saint-Exupéry, Toulouse, France (valentin.kivachuk@irt-saintexupery.com)
  • 2Météo France, Toulouse, France (michael.zamo@meteo.fr)

Any software relies on data, and the meteorological field is not an exception. The importance of using correct and accurate data is as important as using it efficiently. GRIB and NetCDF are the most popular file formats used in Meteorology, being able to store exactly the same data in any of them. However, they differ in how they internally treat the data, and transforming from GRIB (a simpler file format) to NetCDF is not enough to ensure the best efficiency for final applications.

In this study, we improved the performance and storage of ARPEGE cloud cover forecasts post-processing with convolutional neural network and Precipitation Nowcasting using Deep Neural Network projects (proposed in other sessions for the EGU general assembly). The data treatments of both projects were studied and different NetCDF capabilities were applied in order to obtain significantly faster execution times (up to 60 times faster) and more efficient space usage.

How to cite: Kivachuk Burdá, V. and Zamo, M.: NetCDF: Performance and Storage Optimization of Meteorological Data, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-21549, https://doi.org/10.5194/egusphere-egu2020-21549, 2020

Display materials

Display file

Comments on the display material

AC: Author Comment | CC: Community Comment | Report abuse

Display material version 2 – uploaded on 05 May 2020, no comments
Added a Introduction
Display material version 1 – uploaded on 04 May 2020, no comments