AS4.2

In parallel with the new Japanese flagship supercomputer, Fugaku, we have continued improving a nonhydrostatic icosahedral atmospheric model (NICAM). Here, we introduce the results of our system-application co-design since 2014. Fugaku's CPU (A64FX) is based on the Arm instruction-set architecture. This 48-core many-core CPU is equipped with 32GB of HBM2 memory, showing data transfer performance comparable to GPUs. We have implemented kernel-level optimizations to take advantage of Fugaku's high memory performance. Among them, we recognized trade-offs related to ensuring memory locality and parallelism, and register allocation. We improved the application's average arithmetic intensity through detailed loop-by-loop performance measurements and reduced memory pressure by actively using single-precision operations. We also redesigned the data layout and the file I/O component of the ensemble data assimilation (DA) system and achieved good scalability in the atmospheric simulation and DA. We performed a global 3.5km mesh, 1024-member ensemble simulation, and DA using 82% of the Fugaku system (131,072 nodes, 6,291,456 cores). In this world's most massive ensemble DA benchmark experiment, the simulation and the DA achieved 29 PFLOPS and 79 PFLOPS of effective performance.

How to cite: Yashiro, H., Terasaki, K., Kawai, Y., Kudo, S., Miyoshi, T., Imamura, T., Minami, K., Nakano, M., Kodama, C., Satoh, M., and Tomita, H.: The NICAM 3.5km-1024 ensemble simulation: Performance optimization and scalability of NICAM-LETKF on supercomputer Fugaku, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4771, https://doi.org/10.5194/egusphere-egu21-4771, 2021.

The Non-hydrostatic ICosahedral Atmospheric Model (NICAM), a global model with an icosahedral grid system, has been under development for nearly two decades. Here, we present its recent updates for the Coupled Model Intercomparison Project Phase 6, High Resolution Model Intercomparison Project (HighResMIP) and their impact on the simulated mean states using 56-14km mesh model. Major updates include updates of the cloud microphysics scheme and land surface model, introduction of natural and anthropogenic aerosols and a subgrid-scale orographic gravity wave drag scheme, and improvement of the coupling between the cloud microphysics and the radiation schemes. A short-term sensitivity experiments demonstrate improvements in the ice water content, high cloud amount, surface air temperature over the Arctic region, location and strength of zonal mean subtropical jet, and shortwave radiation over Africa and South Asia. The decadal climate simulations further reveal an improvement in the genesis and structure of the tropical cyclones compared with those with the previous model. Finally, we will address outlook toward the cloud-resolving climate simulation based on a fresh benchmark result on supercomputer Fugaku, a flagship supercomputer in Japan.

How to cite: Kodama, C., Yamada, Y., Ohno, T., Seiki, T., Yashiro, H., Noda, A. T., Nakano, M., Roh, W., Satoh, M., Nitta, T., Goto, D., Miura, H., Nasuno, T., Miyakawa, T., Chen, Y.-W., and Sugi, M.: HighResMIP climate simulations with NICAM and beyond on supercomputer Fugaku, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-8580, https://doi.org/10.5194/egusphere-egu21-8580, 2021.

Recent advancement of supercomputing enables us to conduct a climate simulation by using a global model with horizontal grid spacing of a few kilometers. We may need to tune the model in order to conduct a reliable simulation. In order to test feasibility of a few kilometer climate simulation in near future, we conducted one-year simulation from June 2004 to May 2005 by using Nonhydrostatic Icosahedral Atmospheric Model (NICAM) with horizontal grid spacing of 28 km, 14 km, 7 km, and 3.5 km, and evaluated their simulation performances. In general, global models have shown weak wind speed of tropical cyclones compared to its central sea level pressure due to insufficient horizontal resolution. As expected, the 3.5 km simulation showed improvement of this bias. As for simulated mean state, globally annual mean precipitation tended to be decreased with finer horizontal resolution in NICAM. Compared with observation (Global Precipitation Climatology Project V2.2; 2.71 mm day^-1), 7 km and 3.5 km simulations underestimated the global mean precipitation (2.54 mm day^-1 and 2.67 mm day^-1), while 14 km and 28 km simulations overestimated (2.84 mm day^-1 and 2.78 mm day^-1). The 3.5 km simulation showed the best performance for reproducing globally annual mean precipitation. However, the 3.5 simulation showed underestimation of the South Pacific Convergence Zone. In order to conduct a reliable simulation, we need to improve performance of the 3.5 km global model. This demands extensive computing resources. The supercomputer Fugaku will give us extensive computing resources for addressing this issue.

How to cite: Yamada, Y., Kodama, C., Noda, A., Satoh, M., Nakano, M., Miyakawa, T., Yashiro, H., and Nasuno, T.: Evaluating performances of one-year simulation by using 3.5 km mesh global nonhydrostatic model, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-15452, https://doi.org/10.5194/egusphere-egu21-15452, 2021.

Simulation of the Madden-Julian Oscillation (MJO) has been notoriously difficult in atmospheric models. This is partly due to the fact that the reproducibility of the MJO is highly sensitive to parameters that are difficult to fix from observation or theory, and require empirical tuning based on model behaviors. Parameters regards to the cloud-microphysics are some of such parameters that simulations of the MJO are especially sensitive to.

To address this problem, we conducted a set of cloud-microphysics parameter-sweep experiments on a convection-permitting model, NICAM (Nonhydrostatic ICosahedral Atmospheric Model) at 14 km horizontal resolution to seek for a setting which best represents the MJO (MJO-tuned). We then compared the performance of the NICAM in reproducing the MJO using MJO-tuned setting with the standard NICAM setting employed for high resolution model intercomparison project (High Res MIP)-type experiments. The comparison was conducted for 14 km resolution, and for 3.5 km resolution experiments using DYAMOND (DYnamics of the Atmospheric general circulation Modeled On Non-hydrostatic Domains) data, which is based on the MJO-tuned setting.

The comparison indicated that in the 14 km resolutions, the MJO-tuned setting reduces the excessive development of convection over the Maritime Continents which was apparent in the High Res MIP-setting. However, for the 3.5 km experiments convective activities of the MJO appeared to successfully reach the dateline for both the MJO-tuned setting and the High Res MIP-setting. The results of this study implies that a sufficient increase in the horizontal resolution has the potential to reduce the dependency of the microphysics setting on the reproducibility of the MJO, at least in the first few weeks of the simulations on NICAM.

How to cite: Suematsu, T., Yamada, Y., Kodama, C., and Miyakawa, T.: A comparison of the microphysics dependency on the reproducibility of the MJO under different resolutions using NICAM, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-15755, https://doi.org/10.5194/egusphere-egu21-15755, 2021.

The DYAMOND project (DYnamics of the Atmospheric general circulation Modeled On Non-hydrostatic Domains) is the first initiative for a model intercomparison of global storm resolving (km-scale) climate simulations. The analysis of these simulations advances the understanding of the climate system and improves the next-generation of weather and climate models. In a first phase, a period of 40 days from 1st of August 2016 was simulated, with all models starting from the same initial conditions. The resulting data set is referred to as ”DYAMOND Summer” data. In its second, currently ongoing phase ”DYAMOND Winter”, participating models simulate 40 days starting on the 20th of January 2020, also covering the period of the EUREC4A field experiment. While the DYAMOND Summer only included atmosphere models, the DYAMOND Winter data set also includes coupled atmosphere-ocean models resolving ocean-eddies, atmospheric storms and their interactions.
The analysis of these simulations allows to identify robust features common to this class of new models, and provides insights into implementation-dependence of the results and a hint of the future of climate modelling (e.g. Arnold et al., 2020 ; Dueben et al., 2020 ; Stevens et al., 2020 ; Wedi et al., 2020 ). 
The Centre of Excellence in Simulation of Weather and Climate in Europe (ESiWACE) and the German Climate computing centre (DKRZ) are making this data available to the research community. For this purpose, a user-friendly central point of access, the so-called “DYAMOND data library” has been developed. It provides access to the Summer and Winter data collections. A growing community with a lively exchange (e.g. during regular Hackathons) further simplifies the usage of these data sets. 

The presentation will introduce the DYAMOND project with a focus on the new DYAMOND Winter data collection. It will present the corresponding experiment protocol and the participating models. To invite scientists to use these data sets, different ways of using the data on the supercomputer of DKRZ will be described in detail.

How to cite: Duras, J., Ziemen, F., and Klocke, D.: The DYAMOND Winter data collection, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4687, https://doi.org/10.5194/egusphere-egu21-4687, 2021.

In the recent years, global kilometer-scale convection-permitting models have shown promising results in producing realistic convection and precipitation. In this study, a 2.5 km global Icosahedral Nonhydrostatic (ICON) model simulation ran for 40 days (06 UTC 01 Aug – 23 UTC 10 Aug 2016) from Dynamics of the Atmospheric general circulation Modeled On Non-hydrostatic Domains (DYAMOND) initiative was used to identify thermal cold pools (using virtual temperature) over tropical oceans. In addition to examining cold pool variability, variables such as vertical wind shear (0-600 hPa and 0-300 hPa), relative humidity, convective available potential energy (CAPE), column water vapor and surface fluxes corresponding to each cold pool were analyzed. Grid-point linear regression was applied to identify relationships between these variables and cold pool size and intensity. It was found out that there is a statistically significant regional variability in the relationships between cold pool properties and their environments across the global tropics, and cold pool size and intensity have quite different dependence on the various variables considered. Unsupervised machine learning algorithm was then applied to geospatial linear regression to identify coherent patterns explaining multi-modal feedback between cold pools and their mesoscale environments.

Previous studies have hypothesized that although accurate characterization of cold pool diurnal cycle is essential to resolve realistic deep convection in the current generation climate models, our lack of understanding of feedbacks between cold pools and convection leads to distorted diurnal cycle of precipitation. NASA’s RapidScat satellite was in a non-sun-synchronous orbit for 2014-2016 and thus was able to resolve diurnal cycle. Garg et al. (2020) gradient feature technique was applied on RapidScat’s winds to identify cold pools and observe their diurnal cycle of number, size, precipitation and associated convective system properties. Once an observed perspective of cold pool diurnal cycle is obtained, Fourier analysis was used on all the cold pool-associated variables in ICON simulation to obtain the diurnal phase and amplitude. The simulated diurnal cycle of cold pool number, size, precipitation, and other variables were observed to be similar as RapidScat. In this way, this study creates a holistic overview of cold pool-convection-precipitation-storm environment relationships using high-resolution CRM from DYAMOND and satellite observations.

How to cite: Garg, P., Nesbitt, S. W., Lang, T. J., and Priftis, G.: Tropical Oceanic Mesoscale Cold Pools in High-Resolution Global Icosahedral Nonhydrostatic (ICON) Model from DYAMOND, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-7925, https://doi.org/10.5194/egusphere-egu21-7925, 2021.

During the past few years, the Goddard Earth Observing System (GEOS) and Massachusetts Institute of Technology general circulation model (MITgcm) groups have produced, respectively, global atmosphere-only and ocean-only simulations with km-scale grid spacing. These simulations have proved invaluable for process studies and the development of satellite and in-situ sampling strategies. Nevertheless, a key limitation of these simulations is the lack of feedback between the ocean and the atmosphere, limiting their usefulness for studying air-sea interactions and designing observing missions to study these interactions. To remove this limitation, we have coupled the km-scale GEOS atmospheric model with the km-scale MITgcm ocean model. We will present preliminary results from the GEOS-MITgcm contribution to the second phase of the DYAMOND (DYnamics of the Atmospheric general circulation Modeled On Non-hydrostatic Domains) initiative.

The coupled atmosphere-ocean simulation was integrated using a cubed-sphere-1440 (~6-7 km horizontal grid spacing) configuration of GEOS and a lat-lon-cap-2160 (2–5-km horizontal grid spacing) configuration of MITgcm. We will show results from a preliminary analysis of air-sea interactions between Sea Surface Temperature (SST) and surface winds. In particular, we will discuss non-local atmospheric overturning circulation formed above the Gulf Stream SST front with characteristic sub-mesoscale width. This formation of a secondary circulation above the front suggests that capturing such air-sea interaction phenomena requires high-resolution capabilities in both the models' oceanic and atmospheric components.

How to cite: Strobach, E., Molod, A., Trayanov, A., Putman, W., Menemenlis, D., Klein, P., Campin, J.-M., Hill, C., and Henze, C.: GEOS-MITgcm coupled atmosphere-ocean simulation for DYAMOND, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-14947, https://doi.org/10.5194/egusphere-egu21-14947, 2021.

Although the humidity distribution in the tropical free troposphere plays a key role in controlling the Earth’s energy budget, it is poorly simulated in Global Circulation Models (GCMs). A major uncertainty in these models arises from parameterizations of unresolved processes, above all the convective parameterization. An important step in global atmospheric modelling has been made with global storm-resolving models (GSRMs). By forgoing the convective parameterization GSRMs nourish the hope that they better represent processes relevant for humidity, but it is unclear to what extent the uncertainty in free-tropospheric humidity is reduced. The main goal of our study is to quantify this uncertainty as well as the resulting uncertainty in the clear-sky radiation budget based on the spread in an ensemble of GSRMs called DYAMOND. We find that the inter-model spread in relative humidity (RH) in DYAMOND has reduced by at least a factor of two throughout most of the free troposphere compared to the GCMs that participated in the CMIP5 AMIP experiment. However, the remaining RH differences in DYAMOND still cause a considerable inter-model spread of 1.2 Wm^-2 in tropical mean clear-sky outgoing longwave radiation (OLR). For the most part this spread is caused by the RH differences in the lower and mid free troposphere, whereas RH differences in the upper troposphere (above 10 km) have a minor impact on OLR. We only find a direct connection between anomalies in RH and anomalies in the resolved humidity transport in the upper troposphere, suggesting that differences in the parameterizations of unresolved processes like microphysics and turbulence play a major role in the altitude regions with the strongest impact on OLR. Comparing model fields in moisture space, i.e. sorted from the driest to the moistest atmospheric column, reveals that two tropical regimes contribute most to the spread in tropical mean OLR: the driest subsidence regimes and moist regimes at the transition from deep convective to subsidence regions.

How to cite: Lang, T., Naumann, A. K., Stevens, B., and Buehler, S. A.: Inter-model differences in tropical free-tropospheric humidity and their impact on the clear-sky radiation budget in global storm-resolving simulations, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-8227, https://doi.org/10.5194/egusphere-egu21-8227, 2021.

Precipitation observations between March to May 2015 show several coherent propagating systems in an area between 10°N and 10°S with a lifetime of 3-4 weeks demonstrating the importance of simulations beyond a month. The eastward propagation speed is typically 1100 km day^-1. The main origins of significant amounts of precipitation along this belt are the tropical warm pools in the Western Pacific around 158-174°E and the eastern Indian Ocean around 90°E as well as the tropical rainforest over South America around 69°W.

We investigated the lifetime and propagation of tropical precipitating systems based on observations and a near-global convection permitting seasonal simulation with the Weather Research and Forecasting (WRF). The latitude-belt simulation covers an area between 57°S to 65°N with a grid increment of 0.03° over a period of 5 months forced by sea surface temperature (SST) observations.

Results of this simulation with respect to tropical convection were investigated by means of comparison with satellite-based cloud and precipitation observations and ECMWF operational analysis. Wavenumber-frequency spectra of the tropical convection and the detection of various wave pattern were derived from the 3-h outgoing longwave radiation at the top of the atmosphere (TOA OLR) fields and revealed by Wheeler-Kiladis diagrams. The simulation shows the observed spectral signatures of eastward propagating EIGs and Kelvin waves.

The EOF decomposition of the monthly averaged sea level pressure fields demonstrates that 65 % of the sea surface pressure fluctuations in the ECMWF analyses can be explained by the correlation pattern shown in the 1^st EOF. The agreement with the 1^st EOF of the WRF simulation is excellent despite a slight underestimation of the strength of the correlations. The spatial structure is very similar and 61 % of the variance are contained in first EOF. The EOF analyses provided strong evidence that the seasonal simulation with a convection permitting horizontal resolution captures the representation of the teleconnection pattern.

How to cite: Warrach-Sagi, K., Schwitalla, T., and Wulfmeyer, V.: Representation of tropical convection in a near-global convection permitting seasonal simulation with WRF, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-15689, https://doi.org/10.5194/egusphere-egu21-15689, 2021.

Studies have shown the benefits of convection-permitting downscaling at the seasonal scale using limited-area models. To evaluate the performance with real forecasts as boundary conditions, four members of the SEAS5 global ensemble were dynamically downscaled over Ethiopia during June, July, and August 2018 at a 3-km resolution. We used a multi‐physics ensemble based on the WRF model to compare the effects of boundary conditions and physics parametrization producing 16 ensemble members. With ECMWF analyses as a reference, SEAS5 averaged to a +0.17°C bias over Ethiopia whereas WRF resulted in +1.14°C. With respect to precipitation, the WRF model simulated 264 mm compared to 248 mm for SEAS5 and 236 mm for GPM-IMERG. The maximum northward extension of the tropical rain belt decreased by about 2° in both models. Downscaling enhanced the ensemble spread in precipitation by 60% on average, correcting the SEAS5 underdispersion. The WRF ensemble spread over Ethiopia was mostly generated by the perturbed boundary conditions, as their effect is often 50% larger than the physics‐induced variability. The results indicate that boundary condition perturbations are necessary, although not always sufficient, to generate the right amount of ensemble spread in a limited-area model with complex topography. The next step is to use specific methods to calculate the added value provided by the downscaling.

How to cite: Mori, P., Schwitalla, T., Ware, M., Warrach-Sagi, K., and Wulfmeyer, V.: Downscaling of a seasonal ensemble forecast at the convection-permitting resolution over the Horn of Africa using the WRF model, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-12040, https://doi.org/10.5194/egusphere-egu21-12040, 2021.

According to their thermal structure and dynamics, different types of tropospheric cyclones can be defined. Subtropical cyclones (STC) are low pressure systems that share tropical and extratropical characteristics, having a hybrid thermal structure. The impacts of this kind of cyclones are typically like the ones due to tropical storms or even hurricanes, leading to widespread social damage and significant economic losses. Moreover, because of its complex dynamics and rapid intensification, these systems remain a phenomenon of interest, as well as a challenge in terms of prediction. Consequently, effective numerical model simulations become the key tool in order to reliably forecast these extreme events. In this study, a STC event, which occurred in October 2014 nearby the Canary Islands, is assessed by means of the high-resolution numerical weather prediction model HARMONIE-AROME, which is currently operated at 2.5 km grid resolution. This model was developed in the framework of the collaboration of the ten European National Meteorological Services that belong to the HIRLAM international research consortium, together with the sixteen countries that comprise the ALADIN consortium. To evaluate the performance of the simulation, airport observations and sounding data in the vicinity of the STC are considered for local analyses, and satellite images are used to assess the global cloudiness arrangement.

How to cite: Sastre, M., Díaz Fernández, J., Quitián Hernández, L., Bolgiani, P., Santos-Muñoz, D., González-Alemán, J. J., Valero, F., Sebastián-Martín, L. I., López, L., Farrán, J. I., and Martín, M. L.: Simulation of a subtropical cyclone using the HARMONIE-AROME model, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-15665, https://doi.org/10.5194/egusphere-egu21-15665, 2021.

The NASA Global Earth Observing System (GEOS) model supports an array of complex Earth system simulation and assimilation capabilities.  These range from simple development frameworks such as dry atmosphere dynamics and single column physics cases, to fully coupled atmosphere-ocean-land-cryosphere-chemistry. Efficient use of available computational resources requires extensive scientific development within each of these components, and optimized frameworks for coupling and executing these components in a comprehensive manner.  Ultimately, experiment design requires a compromise between complexity and increased resolution.  This talk will explore these compromises within the array of global DYAMOND Phase II winter 40-day simulations completed with GEOS. These include: 1) A coupled 4km ocean and 6km atmosphere with interactive two-moment aerosol cloud microphysics. 2) A 3km 181-level atmosphere with single-moment 6-phase cloud microphysics including 1km global carbon emissions for chemistry transport. 3) A 1.5km 181-level atmosphere with simple parameterized chemistry.

How to cite: Putman, W.: Overcoming the challenges of increasing resolution and complexity in GEOS, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-12782, https://doi.org/10.5194/egusphere-egu21-12782, 2021.

The increase in the capability of Earth System Models (ESMs) is strongly linked to the amount of computing power, given that the spatial resolution used for global climate experimentation is a limiting factor to correctly reproduce climate mean state and variability. However, higher spatial resolutions require new High Performance Computing (HPC) platforms, where the improvement of the computational efficiency of ESMs will be mandatory. In this context, porting a new ultra-high resolution configuration into a new and more powerful HPC cluster is a challenging task, involving technical expertise to deploy and improve the computational performance of such a novel configuration.

To take advantage of this foreseeable landscape, the new EC-Earth 4 climate model is being developed by coupling OpenIFS 43R3 and NEMO 4 as atmosphere and ocean components respectively. An important effort has been made to improve the computational efficiency of this new EC-Earth version, such as extending the asynchronous I/O capabilities of the XIOS server to OpenIFS. 

In order to anticipate the computational behaviour of EC-Earth 4 for new pre-exascale machines such as the upcoming MareNostrum 5 of the Barcelona Supercomputing Center (BSC), OpenIFS and NEMO models are therefore benchmarked on a petascale machine (MareNostrum 4) to find potential computational bottlenecks introduced by new developments or to investigate if previous known performance limitations are solved. The outcome of this work can also be used to efficiently set up new ultra-high resolutions from a computational point of view, not only for EC-Earth, but also for other ESMs.

Our benchmarking consists of large strong scaling tests (tens of thousands of cores) by running different output configurations, such as changing multiple XIOS parameters and number of 2D and 3D fields. These very large tests need a huge amount of computational resources (up to 2,595 nodes, 75 % of the supercomputer), so they require a special allocation that can be applied once a year.

OpenIFS is evaluated with a 9 km global horizontal resolution (Tco1279) and using three different output data sets: no output, CMIP6-based fields and huge output volume (8.8 TB) to stress the I/O part. In addition, different XIOS parameters, XIOS resources, affinity, MPI-OpenMP hybridisation and MPI library are tested. Results suggest new features introduced in 43R3 do not represent a bottleneck in terms of performance as the model scales. The I/O scheme is also improved when outputting data through XIOS according to the scalability curve.

NEMO is scaled using a 3 km global horizontal resolution (ORCA36) with and without the sea-ice module. As in OpenIFS, different I/O configurations are benchmarked, such as disabling model output, only enabling 2D fields, or either producing 3D variables on an hourly basis. XIOS is also scaled and tested with different parameters. While NEMO has good scalability during the most part of the exercise, a severe degradation is observed before the model uses 70% of the machine resources (2,546 nodes). The I/O overhead is moderate for the best XIOS configuration, but it demands many resources.

How to cite: Yepes-Arbós, X., Castrillo, M., C. Acosta, M., and Serradell, K.: Anticipating the computational performance of Earth System Models for pre-exascale systems, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-7888, https://doi.org/10.5194/egusphere-egu21-7888, 2021.

FESOM-2 is a finite volume ocean circulation and sea ice model developed by the Alfred Wegener Institute (AWI). It solves the primitive equations using the hydrostatic and Bousinessq approximations on an unstructured grid, allowing seamless mesh resolution increase towards eddy-resolving scales in regions of high variability or along coast lines. FESOM-2 is a highly optimized MPI-parallel Fortran code that displays excellent scaling to tens of thousands of cores. In the context of ESiWACE-2 services, we have explored the benefits of GPU acceleration of FESOM-2 in a six-month engineering effort. We have determined the flux-corrected tracer transport, and in particular the advection of temperature and salinity, to be a dominant factor in the application profile and we have ported this routine to GPUs using both OpenACC and CUDA-C. We conclude that the memory access patterns in FESOM-2 are suitable to map onto GPU accelerators and that both strategies are viable options, giving significant speedups for tracer advection in high-resolution mesh configurations. We have benchmarked the ported application on Nvidia Kepler, Volta and Ampere architectures and observe that our tuned kernels can approach the peak memory bandwidth, and we also see that OpenACC offers a competitive performance with less development and maintenance effort. We conclude that an expansion of the OpenACC directives is the most promising road to utilize upcoming GPU-equipped exascale machines for FESOM-2.

How to cite: van den Oord, G., Sclocco, A., Moulard, G.-E., Guibert, D., Sidorenko, D., Koldunov, N., van Werkhoven, B., Raffin, E., and Rakowski, N.: GPU acceleration of the FESOM-2 ocean and sea-ice model, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-11551, https://doi.org/10.5194/egusphere-egu21-11551, 2021.

This presentation will give an overview about an ongoing collaboration between the European Centre for Medium-Range Weather Forecasts (ECMWF) and the Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research (AWI). Our recent development is a single-executable coupled configuration of the Integrated Forecasting System (IFS) and the Finite Volume Sea Ice-Ocean Model, FESOM2. This configuration is set up to participate in the DYAMOND project alongside ECMWF’s default IFS-NEMO configuration. IFS-FESOM2 and IFS-NEMO are tentative models to generate “Digital Twin” storm-scale, coupled simulations as envisioned in the European Destination Earth (DestinE) and Next Generation Earth Modelling Systems (NextGEMS) projects.

FESOM2 has a novel dynamical core that supports multi-resolution triangular grids. The model and its predecessor FESOM1 have been used in many studies over the last decade, with a focus on the role of the polar regions in global ocean circulation. The impact of eddy-permitting and locally eddy-resolving resolution has been addressed in CMIP6 and HighResMIP simulations as part of the AWI-CM-1-1 global climate model, while simulations with up to 1km resolution in the Arctic Ocean have been performed in stand-alone mode.

Initially, two coupled IFS-FESOM2 configurations have been tested: A coarse-resolution setup with a nominal 1° ocean, and a DYAMOND-II configuration with 0.25° ocean and IFS at 4.5km global resolution on average. For the latter configuration, FESOM2 is mimicking the “ORCA025” tri-polar curvilinear grid of the NEMO model, whose grid boxes have been split into triangles. Initialisation is from ECMWF’s analysis for IFS and NEMO, and from an ERA5-forced ocean spin-up for FESOM2. We discuss technical challenges with respect to the hybrid OpenMP and MPI parallelization in a single-executable context, describe a novel strategy for resource-efficient writing of model output, and summarise future applications such as exploring the impact of flexible FESOM2 grid configurations on the atmosphere - with ocean simulations that resolve leads in sea ice and ocean eddies almost everywhere.

How to cite: Rackow, T., Wedi, N., Mogensen, K., Dueben, P., Goessling, H. F., Hegewald, J., Kühnlein, C., Zampieri, L., and Jung, T.: DYAMOND-II simulations with IFS-FESOM2, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-9672, https://doi.org/10.5194/egusphere-egu21-9672, 2021.

Earth-System models traditionally use double-precision, 64 bit floating-point numbers to perform arithmetic. According to orthodoxy, we must use such a relatively high level of precision in order to minimise the potential impact of rounding errors on the physical fidelity of the model. However, given the inherently imperfect formulation of our models, and the computational benefits of lower precision arithmetic, we must question this orthodoxy. At ECMWF, a single-precision, 32 bit variant of the atmospheric model IFS has been undergoing rigorous testing in preparation for operations for around 5 years. The single-precision simulations have been found to have effectively the same forecast skill as the double-precision simulations while finishing in 40% less time, thanks to the memory and cache benefits of single-precision numbers. Following these positive results, other modelling groups are now also considering single-precision as a way to accelerate their simulations.

In this presentation I will present the rationale behind the move to lower-precision floating-point arithmetic and up-to-date results from the single-precision atmospheric model at ECMWF, which will be operational imminently. I will then provide an update on the development of the single-precision ocean component at ECMWF, based on the NEMO ocean model, including a verification of quarter-degree simulations. I will also present new results from running ECMWF's coupled atmosphere-ocean-sea-ice-wave forecasting system entirely with single-precision. Finally I will discuss the feasibility of even lower levels of precision, like half-precision, which are now becoming available through GPU- and ARM-based systems such as Summit and Fugaku, respectively. The use of reduced-precision floating-point arithmetic will be an essential consideration for developing high-resolution, storm-resolving Earth-System models.

How to cite: Hatfield, S., Mogensen, K., Dueben, P., Wedi, N., and Diamantakis, M.: Operational Single-Precision Earth-System Modelling at ECMWF, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-733, https://doi.org/10.5194/egusphere-egu21-733, 2021.

Within the ESiWACE2 project we parallelized and optimized OBLIMAP. OBLIMAP is a climate model - ice sheet model coupler that can be used for offline and online coupling with embeddable mapping routines. In order to anticipate future demand concerning higher resolution and/or adaptive mesh applications, a parallel implementation of OBLIMAP's fortran code with MPI has been developed. The data intense nature of this mapping task, required a shared memory approach across the processors per compute node in order to prevent that the node memory is the limiting bottleneck. Besides, the current parallel implementation allows multi node scaling and includes parallel netcdf IO in addition with loop optimizations. Results show that the new parallel implementation offers better performance and scales well. On a single node, the shared memory approach allows now to use all the available cores, up to 128 cores in our experiments on Antarctica 20x20km test case where the original code was limited to 64 cores on this high-end node and it was even limited to 8 cores on moderate platforms. The multi node parallelization offers on Greenland 2x2km test case a speedup of 4.4x on 4 high-end compute nodes equipped with 128 cores each compared to the original code which was able to run only on 1 node. This paves the way to the establishment of OBLIMAP as an candidate ice sheet coupling library candidate for large-scale, high-resolution climate modeling.

How to cite: Raffin, E., Guibert, D., and Reerink, T.: OBLIMAP parallelization and optimization toward high resolution climate model - ice sheet model coupler, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-9880, https://doi.org/10.5194/egusphere-egu21-9880, 2021.

Urban areas make up only a small fraction of the Earth’s surface; however, they are home to over 50% of the global population. Accurate numerical weather prediction (NWP) forecasts in these areas offer clear societal benefits; however, land-atmosphere interactions are significantly different between urban and non-urban environments. Forecasting urban weather requires higher model resolution than the size of the urban domain, which is often achievable by regional but not global NWP models. Here we present the preliminary implementation of an urban scheme within the land surface component of the global Integrated Forecasting System (IFS), at recently developed ~1km horizontal resolution. We evaluate the representation error of fluxes and NWP variables at coarser resolutions (~9 km and ~31 km), using the high resolution as truth. We evaluate the feasibility of the scheme and its urban representation at ~1km scales. Availability of urban mapping data limit the affordable complexity of the global scheme; however, using generalisations model performance is improved over urban sites, even adopting simple schemes, and the modelled Urban Heat Island effects show broad agreement with observations. Several directions for future work are explored including a more complex urban representation, restructuring of the urban tiling and the introduction of an urban emissions model for trace gas emissions.

How to cite: McNorton, J., Bousserez, N., Arduini, G., Agusti-Panareda, A., Balsamo, G., Boussetta, S., Choulga, M., Hadade, I., and Hogan, R.: Global NWP Modeling of urban environments at ~1km resolution, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-7542, https://doi.org/10.5194/egusphere-egu21-7542, 2021.

The ECMWF operational land surface model, based on the Carbon-Hydrology Tiled ECMWF Scheme for Surface Exchanges over Land (CHTESSEL) is the baseline for global weather, climate and environmental applications at ECMWF. In order to expedite its progress and benefit from international collaboration, an ECLand platform has been designed to host advanced and modular schemes. ECLand is paving the way toward a land model that could support a wider range of modelling applications, facilitating global kilometer scales testing as envisaged in the Copernicus and Destination Earth programmes. This presentation introduces the CHTESSEL and its recent new developments that aims at hosting new research applications.

These new improvements touch upon different components of the model: (i) vegetation, (ii) snow, (iii) soil hydrology, (iv) open water/lakes (v) rivers and (vi) urban areas. The developments are evaluated separately with either offline simulations or coupled experiments, depending on their level of operational readiness, illustrating the benchmarking criteria for assessing process fidelity with regards to land surface fluxes and reservoirs involved in water-energy-carbon exchange, and within the Earth system prediction framework, as foreseen to enter upcoming ECMWF operational cycles.

Reference: Souhail Boussetta, Gianpaolo Balsamo*, Anna Agustì-Panareda, Gabriele Arduini, Anton Beljaars, Emanuel Dutra, Glenn Carver, Margarita Choulga, Ioan Hadade, Cinzia Mazzetti, Joaquìn Munõz-Sabater, Joe McNorton, Christel Prudhomme, Patricia De Rosnay, Irina Sandu, Nils Wedi, Dai Yamazaki, Ervin Zsoter, 2021: ECLand: an ECMWF land surface modelling platform, MDPI Atmosphere, (in prep).

How to cite: Balsamo, G. and Boussetta, S.: ECLand: an ECMWF land surface modelling platform, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-16240, https://doi.org/10.5194/egusphere-egu21-16240, 2021.

Traditional global climate models (GCMs) with coarse uniform resolution (UR) usually have deficiency in simulating realistic results at regional scale, while experimental global high-resolution models show benefits but also raise much computational burden. In recent years, variable resolution (VR) models with unstructured mesh are found to provide comparable results at regional scale and require less computational resources. In this study, the variable resolution CAM-MPAS model with the MPAS (Model for Prediction Across Scales) dynamical core coupled with CAM5 (Community Atmosphere Model Version 5) physics package is used to evaluate the effect of 30 km regional refinement over East Asia on the precipitation simulation. Our results show that the CAM-MPAS model can reasonably reproduce the annual and seasonal precipitation over East Asia, and the MPAS-VR simulation shows reduced mean bias and improvements in seasonal cycle, intensity distribution, and interannual variation compared with the low resolution MPAS-UR simulation. Furthermore, the major contribution to the improvements over the Tibet Plateau in the MPAS-VR experiment comes from the increase of the grid spacing rather than the terrain resolution.

How to cite: Liang, Y., Yang, B., Wang, M., Tang, J., Sakaguchi, K., and Leung, L. R.: Multiscale Simulation of Precipitation over East Asia by Variable Resolution CAM-MPAS, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-1713, https://doi.org/10.5194/egusphere-egu21-1713, 2021.

The local climate of Southern Patagonia is strongly influenced by the interaction between the topography and persistent westerlies, which can generate föhn events. These events are characterized by warm, dry and windy conditions in the lee together with an increase in solar radiation, all factors that can strengthen local glacier ablation. The upstream flow regime influences the generation of different types of föhn which dictate the lee-side atmospheric response regarding the strength, spatial extent and phenomenology. On the basis of the phenomenological character of föhn types, inferences may be drawn on the spatial variability of the local climate and thus also on the glacier impact.

We use a combination of observational data from four automatic weather stations (AWSs) and high-resolution numerical modeling with the Weather Research and Forecasting (WRF) model for a region in Southern Patagonia (48° S–52° S, 72° W–76.5° W) including the Southern Patagonian Icefield (SPI). In order to resolve the complex topography of the region, the model was set up using one-way grid nesting over three domains, which increases the resolution from 20 km in the parent domain to 4 km and 1 km in the nested domains. The final model parametrizations were selected based on 20 sensitivity runs. Evaluation of WRF model surface variables against the AWS data showed that overall the atmospheric fields and the föhn signals were well reproduced, however we found an overestimation in wind velocities.

The application of a föhn identification algorithm to a 10-month study period (June 2018–March 2019) reveals 81 föhn events in total. A simulation of three events of differing flow regimes (supercritical, subcritical, transition) suggests that a supercritical flow regime leads to a linear föhn event with a large spatial extent but moderate intensity. In contrast, a spatially limited but locally strong föhn response is induced by a subcritical regime with upstream blocking and by a transition regime with a hydraulic jump present. Our results imply that the hydraulic jump-type föhn event (transition case) is the most critical for glacier mass balances since it shows the strongest warming, drying, wind velocities and largest solar radiation increases over the SPI. The consideration of flow regimes over the last 40 years shows that subcritical flow occurs most frequently (78%), however transitional flow occurs 14% of the time, implying the potential impact on Patagonian glaciers.

How to cite: Temme, F., Turton, J., Mölg, T., and Sauter, T.: How föhn events of different type characterize the local climate of Southern Patagonia, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-4074, https://doi.org/10.5194/egusphere-egu21-4074, 2021.

While large-domain simulations without convective parameterization are now computationally feasible, microphysics, particularly that of the ice phase, remains a persistent problem for high-resolution models. In 2.5-km equivalent resolution simulations with the ICON model, we find that switching between one- and two-moment ice microphysics can alter cloud top cooling by a factor of ten and in-cloud heating by a factor of four above 350 hPa. A consistent ice crystal effective radius between microphysics and radiation increases the cloud-radiative heating another two-fold, while inclusion of aerosol-cloud interactions reduces it at lower levels between 400 and 500 hPa. We also generate 60-hour trajectories from ICON within ice clouds and use them to force a detailed ice microphysics box model, the Chemical Lagrangian Model of the Stratosphere (ClaMs-ice). We compare the ice mass and number tendencies, as well as the sedimentation fluxes, between ICON and CLaMS-ice. These offline simulations also allow us to quantify the strength of microphysical-radiative feedbacks and investigate the impact on heating of particular ice microphysical factors, including gravity wave parameterization, ice-nucleating particle concentrations, and the number concentration of solution droplets.

How to cite: Sullivan, S., Voigt, A., Miltenberger, A., Rolf, C., and Krämer, M.: How much variability in upper tropospheric cloud-radiative heating can be attributed to ice microphysics?, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-5732, https://doi.org/10.5194/egusphere-egu21-5732, 2021.

Tropical instability waves (TIWs) are oceanic cusp-like features propagating westward along the northern front of the tropical pacific cold tongue. Observational and modelling studies suggest that TIWs may have a large impact on the eastern tropical Pacific background state from seasonal to interannual time-scales, through heat advection and mixing. However, observations are coarse or limited to surface data, and modelling studies are often based on the comparison of low- vs. high-resolution simulations. In this study, we perform a set of regional high-resolution ocean simulations (CROCO 1/12°) in which we strongly damp (NUDG-RUN) or not (CTRL-RUN) TIWs propagation, by nudging the mixed layer meridional current velocities in the TIWs active region toward their climatological values. We first show that this approach do not alter the model internal physics, in particular related to the equatorial wave dynamics. The impact of TIWs on the oceanic mean state (zonal current and heat budget) is then assessed by comparing CTRL-RUN to NUDG-RUN. This approach allows quantifying for the first time the rectified effect of TIWs without degrading the model horizontal resolution, and may lead to a better understanding of ENSO asymmetry and the development of accurate TIWs parameterizations in Earth system models.

How to cite: Maillard, L., Boucharel, J., and Renault, L.: Effect of tropical instability waves on the eastern tropical Pacific basin: damping of TIWs in a high-resolution ocean circulation model, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-7802, https://doi.org/10.5194/egusphere-egu21-7802, 2021.

We examine the representation of Föhn events across the Antarctic Peninsula Mountains during 2011 as they were observed in measurements by an Automatic Weather Station, and in simulations with the Weather Research and Forecasting Model (WRF) as run for the Antarctic Mesoscale Prediction System (AMPS). On the Larsen Ice Shelf (LIS) in the lee of this mountain range Föhn winds are thought to provide the atmospheric conditions for significant warming over the ice shelf thus leading to the initial firn densification and subsequently providing the melt water for hydrofracturing. This process has led to the dramatic collapse of huge parts of the LIS in 1995 and 2002 respectively.

Measurements obtained at a crest AWS on the Avery Plateau (AV), and the analysis of conditions upstream using the Froude number help to put observations at CP into a wider context. We find that, while the model generally simulates meteorological parameters very well, and shows good skills in capturing the occurrence, frequency and duration of Föhn events realistically, it underestimates the temperature increase and the humidity decrease during the Föhn significantly, and may thus underestimate the contribution of Föhn to driving surface melt on the LIS.

Our results indicate that the misrepresentation of cloud properties and particularly the absence of mixed phase clouds in AMPS, affects the quality of weather simulation under normal conditions to some extent, and to a larger extent the model’s capability to simulate the strength of Föhn conditions - and thus their contribution to driving surface melt on the LIS - adequately. Most importantly our data show that Föhn conditions can raise the air temperature to above freezing, and thus trigger melt/sublimation even in winter.

How to cite: Kirchgaessner, A., King, J., Gadian, A., and Anderson, P.: Simulations of Föhn in Antarctica with WRF for the Antarctic Mesoscale Prediction System AMPS, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-9601, https://doi.org/10.5194/egusphere-egu21-9601, 2021.

Rainfall in Africa is difficult to estimate accurately due to the large spatial variability. Most of the monsoon rainfall is generated by convective rainstorms that can be very localized, sometimes covering less than 100 km2. The goal of the African Rainfall Project is to run the Weather and Research Forecast (WRF) model for sub-Saharan Africa at a convection-permitting resolution in order to better represent such rainfall events. The resolution will be 1km, which is finer than most studies over Africa, which typically use resolutions of 3km or more. Running WRF for such a large area at such a high resolution is computationally expensive, which is where IBM’s World Community Grid comes in. The World Community Grid (WCG) is part of the Social Corporate Responsibility of IBM that crowdsources unused computing power from volunteers devices and donates it to scientific projects.

The simulation was adapted to the WCG by dividing the simulation of one year over sub-Saharan Africa in many smaller simulations of 48h over 52 by 52 km domains. These simulations are small enough to be calculated on a single computer of a volunteer at the required resolution. In total, 35609 overlapping domains are covering the whole of sub-Saharan Africa. During the post-processing phase, the smaller simulations are merged back together to obtain one consistent simulation over the whole continent.

Our main focus is rainfall, as this is the variable with the highest socio-economic impact in Africa. However, the outputs of the simulations include other variables such as the 2m-temperature, the 10m-wind speed and direction. These variables are outputted every 15min. At the end of this project, we will have over 3 billion files for a total of 0.5 PB. The data will be reorganized so that the different variables can be stored, searched and retrieved efficiently. After the reorganization, the data will be made publicly available.

The first validation step will be to examine the impact of dividing sub-Saharan Africa into many smaller domains. This will be done by comparing the simulation from this project to one large simulation. This simulation is obtained by running WRF at a 1km resolution on a large domain (500km by 1000km) for a shorter period, using Cartesius, the Dutch national computer. The second validation step will be to compare the simulations with satellite data and with in-situ measurements from the TAHMO network (www.tahmo.org).

How to cite: Le Coz, C., Yu, Q., Treinish, L. A., Alvarez, M. G., Cryan, A., and van de Giesen, N.: High-resolution weather simulation for sub-Saharan Africa on the World Community Grid, EGU General Assembly 2021, online, 19–30 Apr 2021, EGU21-12281, https://doi.org/10.5194/egusphere-egu21-12281, 2021.