EGU22-8094, updated on 08 Apr 2024
https://doi.org/10.5194/egusphere-egu22-8094
EGU General Assembly 2022
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.

Approach to make an I/O server performance-portable across different platforms: OpenIFS-XIOS integration as a case study

Jan Streffing1, Xavier Yepes-Arbós2, Mario C. Acosta2, and Kim Serradell2
Jan Streffing et al.
  • 1Alfred-Wegener-Institut, Helmholtz-Zentrum für Polar- und Meeresforschung, Bremerhaven, Germany
  • 2Barcelona Supercomputing Center, Earth Sciences, Barcelona, Spain

Current Earth System Models (ESMs) produce a large amount of data due to an increase in the simulated complexity of the models and their demanded rising spatial resolution. With the exascale era approaching rapidly, an efficient I/O will be critical to sustain model throughput. The most commonly adopted approach in ESMs is the use of scalable parallel I/O solutions that are intended to minimize the overhead of writing data into the storage system. However, I/O servers with inline diagnostics introduce more complexity and many parameters that need to be tuned. This means that it is necessary to achieve an optimal trade-off between throughput and resource usage.

ESMs are usually run on different platforms which might have different architectural specifications: latency, bandwidth, number of cores and memory per node, file system, etc. In addition, a single ESM can run different configurations which require different amounts of resources, resolution, output frequency, number of fields, etc. Since each individual case is particular, the I/O server should be tuned accordingly to each platform and model configuration.

We present an approach to identify and tune a series of important parameters that should be considered in an I/O server. In particular, we focus on the XML Input/Output Server (XIOS) and we use it integrated with OpenIFS –an atmospheric general circulation model– as a case study. We do not only tune basic parameters such as number of XIOS servers, number of servers per node, type and frequency of post-processing operations, etc., but also specific ones such as XIOS buffer size, splitting of NetCDF files across I/O servers, Lustre striping, 2-level server mode of XIOS, etc.

The evaluation of different configurations on different machines proves that it is possible and necessary to find a proper setup for XIOS to achieve a good throughput using an adequate consumption of computational resources. In addition, the results show that the OpenIFS-XIOS integration is performant on the platforms evaluated. This suggests that the integration is portable, though it was initially developed for a specific platform.

How to cite: Streffing, J., Yepes-Arbós, X., C. Acosta, M., and Serradell, K.: Approach to make an I/O server performance-portable across different platforms: OpenIFS-XIOS integration as a case study, EGU General Assembly 2022, Vienna, Austria, 23–27 May 2022, EGU22-8094, https://doi.org/10.5194/egusphere-egu22-8094, 2022.

Displays

Display file