Analysing and modelling the performance of the HemeLB lattice-Boltzmann simulation environment

https://doi.org/10.1016/j.jocs.2013.03.002Get rights and content
Under a Creative Commons license
open access

Highlights

  • We investigate the scalability of the HemeLB blood flow simulation environment.

  • HemeLB scales well up to 32,768 cores, even when applied to sparse geometries.

  • HemeLB supports interactive steering and visualisation at a frame rate up to 10.6.

  • Our performance model allows advance estimates of the simulation completion time.

Abstract

We investigate the performance of the HemeLB lattice-Boltzmann simulator for cerebrovascular blood flow, aimed at providing timely and clinically relevant assistance to neurosurgeons. HemeLB is optimised for sparse geometries, supports interactive use, and scales well to 32,768 cores for problems with ∼81 million lattice sites. We obtain a maximum performance of 29.5 billion site updates per second, with only an 11% slowdown for highly sparse problems (5% fluid fraction). We present steering and visualisation performance measurements and provide a model which allows users to predict the performance, thereby determining how to run simulations with maximum accuracy within time constraints.

Keywords

Lattice-Boltzmann
Parallel computing
High-performance computing
Performance modelling

Cited by (0)

Derek Groen is a post-doctoral research associate in the CCS at University College London, specialised in multiscale simulation and parallel/distributed computing. He has worked with a wide range of applications, including those using lattice-Boltzmann, N-body and molecular dynamics methods, and participated in several EU-funded IT projects. He finished his PhD in 2010 at the University of Amsterdam, where he investigated the performance of N-body simulations run across geographically distributed (supercomputing) infrastructures. Derek currently works on constructing and testing multiscale simulation applications of cerebrovascular blood flow and clay–polymer nanocomposites.

James Hetherington is a research software engineer, combining computational science research experience with professional software development skills. He helps UCL to produce maintainable, usable, well-tested scientific software which with a lasting impact, bringing software engineering best practice into computational research. At AMEE UK Limited, he developed systems to make it easier for organisations to understand their environmental impacts, creating AMEE Explorer, winner of a Best of What's New award in Popular Science Magazine in 2010. He has also worked on software for model management at the MathWorks, systems biology at UCL CoMPLEx, and particle physics at the Cavendish Laboratory, Cambridge.

Hywel B. Carver received his MEng degree in Information Engineering from King's College, Cambridge, with Distinction in 2008. He is now a second-year PhD student, interested in using high-performance computing as a predictive tool in solving clinical problems. His current focus is the application of lattice-Boltzmann to intracranial haemodynamics, as a means of predicting rupture risks in neurovascular aneurysms.

Rupert W. Nash completed his doctoral studies in the School of Physics at the University of Edinburgh in 2010, studying simple models of swimming particles using lattice-Boltzmann methods. He has since been at the Centre for Computational Science at UCL, working on computational haemodynamics. His research interests are focussed on lattice-Boltzmann methods and high performance computing.

Miguel O. Bernabeu received his DPhil in computational biology from the University of Oxford, UK, in 2011 and his MSc and BEng from the Universidad Politécnica de Valencia, Spain, in 2005 and 2007. He is currently a 2020 Science Research Fellow at the Centre for Computational Science and CoMPLEX, University College London, UK. He has worked in various UK and EU projects developing HPC software for different aspects of cardiovascular modelling and simulation. His research interests include software engineering, parallel computing, and numerical methods with applications to cardiac electrophysiology and brain haemodynamics.

Prof. Peter V. Coveney holds a Chair in physical chemistry and is Director of the Centre for Computational Science, and an Honorary Professor in Computer Science. He is also professor adjunct within the Medical School at Yale University, and Director of the UCL Computational Life and Medical Sciences Network. Coveney is active in a broad area of interdisciplinary theoretical research including condensed matter physics and chemistry, materials science, life and medical sciences. He has published over 300 papers, books and edited works, including the acclaimed bestsellers The Arrow of Time and Frontiers of Complexity, both with Roger Highfield.