Dynamic Core Binding for Load Balancing of Applications Parallelized with MPI/OpenMP

Abstract

Load imbalance is a critical problem that degrades the performance of parallelized applications in massively parallel processing. Although an MPI/OpenMP implementation is widely used for parallelization, users must maintain load balancing at the process level and thread (core) level for effective parallelization. In this paper, we propose dynamic core binding (DCB) to processes for reducing the computation time and energy consumption of applications. Using the DCB approach, an unequal number of cores is bound to each process, and load imbalance among processes is mitigated at the core level. This approach is not only improving parallel performance but also reducing power consumption by reducing the number of using cores without increasing the computational time. Although load balancing among nodes cannot be handled by DCB, we also examine how to solve this problem by mapping processes to nodes. In our numerical evaluations, we implemented a DCB library and applied it to the lattice $\mathcal {H}$-matrixes. Based on the numerical evaluations, we achieved a 58% performance improvement and 77% energy consumption reduction for the applications using the lattice $\mathcal {H}$-matrix.