EGU2020-9686, updated on 12 Jan 2021
https://doi.org/10.5194/egusphere-egu2020-9686
EGU General Assembly 2020
© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Python-based Multidimensional and Parallel Climate Model Data Analysis in ECAS

Regina Kwee1, Tobias Weigel1, Hannes Thiemann1, Karsten Peters1, Sandro Fiore2, and Donatello Elia2
Regina Kwee et al.
  • 1DKRZ, Data Managment, Germany (kwee@dkrz.de)
  • 2CMCC

This contribution highlights the Python xarray technique in context of a climate specific application (typical formats are NetCDF, GRIB and HDF).

We will see how to use in-file metadata and why they are so powerful for data analysis, in particular by looking at community specific problems, e.g. one can select purely on coordinate variable names. ECAS, the ENES Climate Analytics Service available at Deutsches Klimarechenzentrum (DKRZ), will help by enabling faster access to the high-volume simulation data output from climate modeling experiments. In this respect, we can also make use of “dask” which was developed for parallel computing and can smoothly work with xarray. This is extremely useful when we want to exploit fully the advantages of our supercomputer.

Our fully integrated service offers an interface via Jupyter notebooks (ecaslab.dkrz.de). We provide an analysis environment without the need of costly transfers, accessing CF standardized data files and all accessible via the ESGF portal on our nodes (esgf-data.dkrz.de). We can analyse the data of e.g. CMIP5, CMIP6, Grand Ensemble and observation data. ECAS was developed in the frame of European Open Source Cloud (EOSC) hub.

How to cite: Kwee, R., Weigel, T., Thiemann, H., Peters, K., Fiore, S., and Elia, D.: Python-based Multidimensional and Parallel Climate Model Data Analysis in ECAS, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-9686, https://doi.org/10.5194/egusphere-egu2020-9686, 2020

Display materials

Display file

Comments on the display material

AC: Author Comment | CC: Community Comment | Report abuse

Display material version 5 – uploaded on 19 May 2020, no comments
adding new plot to illustrate massive data
Display material version 4 – uploaded on 11 May 2020, no comments
final authorlist
Display material version 3 – uploaded on 09 May 2020, no comments
uploading version
Display material version 2 – uploaded on 07 May 2020, no comments
new authorlist
Display material version 1 – uploaded on 06 May 2020, no comments