CHAMELEON: Reactive Load Balancing for Hybrid MPI+OpenMP Task-Parallel Applications

https://doi.org/10.1016/j.jpdc.2019.12.005Get rights and content
Under a Creative Commons license
open access

Highlights

  • Increasing dynamic variability observable in modern hardware and software.

  • Performance prediction and load balancing difficult or even impossible.

  • Novel library for fine-granular task-based load balancing in hybrid MPI+OpenMP codes.

  • Reactive task migration concept based on online performance data.

  • Evaluation shows performance improvements for hardware and work-induced imbalances.

Abstract

Many applications in high performance computing are designed based on underlying performance and execution models. While these models could successfully be employed in the past for balancing load within and between compute nodes, modern software and hardware increasingly make performance predictability difficult if not impossible. Consequently, balancing computational load becomes much more difficult. Aiming to tackle these challenges in search for a general solution, we present a novel library for fine-granular task-based reactive load balancing in distributed memory based on MPI and OpenMP. With our approach, individual migratable tasks can be executed on any MPI rank. The actual executing rank is determined at run time based on online performance data. We evaluate our approach under an enforced power cap and under enforced clock frequency changes for a synthetic benchmark and show its robustness for work-induced imbalances for a realistic application. Our experiments demonstrate speedups of up to 1.31X.

Keywords

Reactivity
Task migration
Hybrid MPI+openMP
Load balancing
Tasking

Cited by (0)

Jannis Klinkenberg is a Ph.D. candidate at the Chair for High Performance Computing at RWTH Aachen University. He received a B.Sc. degree in Scientific Computing from FH/RWTH Aachen in 2010 and a M.Sc. degree in Artificial Intelligence from Maastricht University in 2012. His research interests include runtime improvements for dynamic and heterogeneous systems, parallel and task-based programming models and the optimization of scientific workloads for modern HPC platforms. Since 2016, he is also actively participating in the OpenMP Language Committee.

Philipp Samfass is a Ph.D. candidate at the Chair of Scientific Computing in Computer Science at Technical University of Munich (TUM). He holds a B.Sc. degree in Computer Science from TUM and a M.Sc. degree in Computer Science from the University of Illinois at Urbana–Champaign. His research focuses on developing new algorithmic approaches for upcoming challenges in high performance computing such as load balancing, fault tolerance and hybrid parallel programming. He is funded in part by the BMBF (Chameleon project) and the EU (ExaHyPE project).

Michael Bader is associate professor at the Department of Informatics at Technical University of Munich. He works on hardware-aware algorithms in computational science and engineering and in high performance computing. In particular, he focuses on challenges imposed by latest supercomputing platforms and the development of suitable efficient and scalable algorithms and software for simulation tasks in science and engineering.

Christian Terboven received a Doctor of Natural Sciences degree from RWTH Aachen University, Germany. He leads the HPC Group at RWTH Aachen University, and his research interests include parallel programming models, related software engineering aspects, and the optimization of simulation codes for modern HPC architectures. Since 2006, he serves on the OpenMP Language Committee and is Chair of the Affinity Subcommittee.

Matthias S. Mueller is full professor for High Performance Computing at RWTH Aachen University and head of the Computation and Communication Center at RWTH. His research interests include programming methodologies, software development tools and computational science on high performance computers. He received his Ph.D. in Physics from Stuttgart University in 2001. From 1999 to 2005 he worked at the High Performance Computing Center (HLRS) in Stuttgart, Germany, which he left as a deputy director. From 2005 until 2012 he was deputy director and CTO at the Center for Information Services and High Performance Computing (ZIH) at Technical University of Dresden.