A hybrid parallel DEM approach with workload balancing based on HSFC

The purpose of this paper is to present a methodology of hybrid parallelization applied to the discrete element method that combines message-passing interface and OpenMP to improve computational performance. The scheme is based on mapping procedures based on Hilbert space-filling curves (HSFC).

The methodology uses domain decomposition strategies to distribute the computation of large-scale models in a cluster. It also partitions the workload of each subdomain among threads. This additional procedure aims to reach higher computational performance by adjusting the usage of message-passing artefacts and threads. The main objective is to reduce the communication among processes. The work division by threads employs HSFC in order to improve data locality and to avoid related overheads. Numerical simulations presented in this work permit to evaluate the proposed method in terms of parallel performance for models that contain up to 3.2 million particles.

Distinct partitioning algorithms were used in order to evaluate the local decomposition scheme, including the recursive coordinate bisection method and a topological scheme based on METIS. The results show that the hybrid implementations reach better computational performance than those based on message passing only, including a good control of load balancing among threads. Case studies present good scalability and parallel efficiencies.

The proposed approach defines a configurable execution environment for numerical models and introduces a combined scheme that improves data locality and iterative workload balancing.