SC5.3 | Parallel computing using asynchronous many-tasks
Parallel computing using asynchronous many-tasks
Co-organized by ESSI3/NH12/OS5
Convener: Kor de Jong | Co-convener: Oliver Schmitz
Mon, 24 Apr, 10:45–12:30 (CEST)
 
Room -2.61/62
Mon, 10:45
Due to the continuous increase in geographical data set sizes and the number of computations that have to be performed in numerical modelling or data analyses, there is often a need to improve the performance and scalability of the software used. Developing such software can be challenging.

In this short course we will introduce the asynchronous many-tasks (AMT) approach, which can be used to develop software that performs and scales well over cores in a single computer as well as over nodes in a computer cluster. We will explain the general principles behind AMT, and show how the HPX C++ software library [1] can be used to develop an example algorithm, calculating hill shading from a digital elevation model, in parallel.

One advantage of using the HPX library is that it provides a single high-level API for performing parallel computations on both shared and distributed memory systems. This contrasts with a popular approach of using multiple APIs - and their associated programming models - for these, like OpenMP and MPI.

The HPX library is successfully being used in various HPC applications, one of which is the LUE numerical modelling framework [2, 3, 4]. With LUE model developers can implement their models using Python and execute them unchanged on their laptop or on a computer cluster.

The goal of this short course is to introduce the attendants to the principles behind AMT and the HPX library, and allow them to be able to decide whether the approach is applicable in their own use-cases. The short course is especially relevant for research software engineers, but we welcome everybody interested in the topic.

- [1] HPX website, https://hpx.stellar-group.org
- [2] LUE website, https://lue.computationalgeography.org
- [3] De Jong, K., Panja, D., Van Kreveld, M., Karssenberg, D. (2021), An environmental modelling framework based on asynchronous many-tasks: scalability and usability, Environmental Modelling & Software, doi: 10.1016/j.envsoft.2021.104998
- [4] De Jong, K., Panja, D., Karssenberg, D., Van Kreveld, M. (2022), Scalability and composability of flow accumulation algorithms based on asynchronous many-tasks, Computers & Geosciences, doi: 10.1016/j.cageo.2022.105083