Due to the continuous increase in geographical data set sizes and the number of computations that have to be performed in numerical modelling or data analyses, there is often a need to improve the performance and scalability of the software used. Developing such software can be challenging.
In this short course we will introduce the asynchronous many-tasks (AMT) approach, which can be used to develop software that performs and scales well over cores in a single computer as well as over nodes in a computer cluster. We will explain the general principles behind AMT, and show how the HPX C++ software library [1] can be used to develop an example algorithm, calculating hill shading from a digital elevation model, in parallel.
One advantage of using the HPX library is that it provides a single high-level API for performing parallel computations on both shared and distributed memory systems. This contrasts with a popular approach of using multiple APIs - and their associated programming models - for these, like OpenMP and MPI.
The HPX library is successfully being used in various HPC applications, one of which is the LUE numerical modelling framework [2, 3, 4]. With LUE model developers can implement their models using Python and execute them unchanged on their laptop or on a computer cluster.
The goal of this short course is to introduce the attendants to the principles behind AMT and the HPX library, and allow them to be able to decide whether the approach is applicable in their own use-cases. The short course is especially relevant for research software engineers, but we welcome everybody interested in the topic.
- [1] HPX website, https://hpx.stellar-group.org
- [2] LUE website, https://lue.computationalgeography.org
- [3] De Jong, K., Panja, D., Van Kreveld, M., Karssenberg, D. (2021), An environmental modelling framework based on asynchronous many-tasks: scalability and usability, Environmental Modelling & Software, doi: 10.1016/j.envsoft.2021.104998
- [4] De Jong, K., Panja, D., Karssenberg, D., Van Kreveld, M. (2022), Scalability and composability of flow accumulation algorithms based on asynchronous many-tasks, Computers & Geosciences, doi: 10.1016/j.cageo.2022.105083
Please decide on your access
Please use the buttons below to download the supplementary material or to visit the external website where the presentation is linked. Regarding the external link, please note that Copernicus Meetings cannot accept any liability for the content and the website you will visit.
You are going to open an external link to the asset as indicated by the session. Copernicus Meetings cannot accept any liability for the content and the website you will visit.