Dave A. May
Karl Rupp
ABSTRACT
Elliptic partial differential equations (PDEs) frequently arise in continuum descriptions of physical processes relevant to science and engineering. Multilevel preconditioners represent a family of scalable techniques for solving discrete PDEs of this type and thus are the method of choice for high-resolution simulations. The scalability and time-to-solution of massively parallel multilevel preconditioners can be adversely affected by using a coarse-level solver with sub-optimal algorithmic complexity. To maintain scalability, agglomeration techniques applied to the coarse level have been shown to be necessary.
In this work, we present a new software component introduced within the Portable Extensible Toolkit for Scientific computation (PETSc) which permits agglomeration. We provide an overview of the design and implementation of this functionality, together with several use cases highlighting the benefits of agglomeration. Lastly, we demonstrate via numerical experiments employing geometric multigrid with structured meshes, the flexibility and performance gains possible using our MPI-rank agglomeration implementation.
- M. F. Adams, H. H. Bayraktar, T. M. Keaveny, and P. Papadopoulos. Ultrascalable implicit finite element analyses in solid mechanics with over a half a billion degrees of freedom. In Proceedings of the 2004 ACM/IEEE Conference on Supercomputing, SC '04, pages 34--, Washington, DC, USA, 2004. IEEE Computer Society. Google Scholar
Digital Library
- S. Balay, S. Abhyankar, M. F. Adams, J. Brown, P. Brune, K. Buschelman, L. Dalcin, V. Eijkhout, W. D. Gropp, D. Kaushik, M. G. Knepley, L. C. McInnes, K. Rupp, B. F. Smith, S. Zampini, and H. Zhang. PETSc users manual. Technical Report ANL-95/11 - Revision 3.6, Argonne National Laboratory, 2015.Google Scholar
- S. Balay, S. Abhyankar, M. F. Adams, J. Brown, P. Brune, K. Buschelman, L. Dalcin, V. Eijkhout, W. D. Gropp, D. Kaushik, M. G. Knepley, L. C. McInnes, K. Rupp, B. F. Smith, S. Zampini, and H. Zhang. PETSc Web page. http://www.mcs.anl.gov/petsc, 2015.Google Scholar
- S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith. Efficient management of parallelism in object oriented numerical software libraries. In E. Arge, A. M. Bruaset, and H. P. Langtangen, editors, Modern Software Tools in Scientific Computing, pages 163--202. Birkhäuser Press, 1997. Google ScholarDigital Library
- M. Blatt, O. Ippisch, and P. Bastian. A massively parallel algebraic multigrid preconditioner based on aggregation for elliptic problems with heterogeneous coefficients. arXiv preprint arXiv:1209.0960v2, 2012.Google Scholar
- J. Brown, B. Smith, and A. Ahmadia. Achieving textbook multigrid efficiency for hydrostatic ice sheet flow. SIAM Journal on Scientific Computing, 35(2):B359--B375, 2013.Google ScholarDigital Library
- M. Emans. Coarse-grid treatment in parallel AMG for coupled systems in CFD applications. Journal of Computational Science, 2(4):365--376, 2011.Google ScholarCross Ref
- B. Gmeiner, H. Köstler, M. Stürmer, and U. Rüde. Parallel multigrid on hierarchical hybrid grids: a performance study on current high performance computing clusters. Concurrency and Computation: Practice and Experience, 26(1):217--240, 2014.Google ScholarCross Ref
- B. Gmeiner, U. Rüde, H. Stengel, C. Waluga, and B. Wohlmuth. Performance and scalability of hierarchical hybrid multigrid solvers for Stokes systems. SIAM Journal on Scientific Computing, 37(2):C143--C168, 2015.Google ScholarCross Ref
- B. Gmeiner, U. Rüde, H. Stengel, C. Waluga, and B. Wohlmuth. Towards textbook efficiency for parallel multigrid. Numerical Mathematics: Theory, Methods and Applications, 8(01):22--46, 2015.Google Scholar
- T. Hoefler, J. Dinan, D. Buntinas, P. Balaji, B. Barrett, R. Brightwell, W. Gropp, V. Kale, and R. Thakur. MPI + MPI: a new hybrid approach to parallel programming with MPI plus shared memory. Computing, 95(12):1121--1136, 2013. Google ScholarDigital Library
- T. Isaac, G. Stadler, and O. Ghattas. Solution of nonlinear Stokes equations discretized by high-order finite elements on nonconforming and anisotropic meshes, with application to ice sheet dynamics. SIAM Journal on Scientific Computing, 37(6):B804--B833, 2015.Google ScholarDigital Library
- S. M. Lechmann, D. A. May, B. J. P. Kaus, and S. M. Schmalholz. Comparing thin-sheet models with 3-D multilayer models for continental collision. Geophysical Journal International, 187(1):10--33, 2011.Google ScholarCross Ref
- J. Li and O. B. Widlund. On the use of inexact subdomain solvers for BDDC algorithms. Computer Methods in Applied Mechanics and Engineering, 196(8):1415--1428, 2007.Google ScholarCross Ref
- L. Luo, C. Yang, Y. Zhao, and X.-C. Cai. A scalable hybrid algorithm based on domain decomposition and algebraic multigrid for solving partial differential equations on a cluster of CPU/GPUs. In 2nd International Workshop on GPUs and Scientific Applications (GPUScA 2011), page 45, 2011.Google Scholar
- D. A. May, J. Brown, and L. Le Pourhiet. A scalable, matrix-free multigrid preconditioner for finite element discretizations of heterogeneous Stokes flow. Computer Methods in Applied Mechanics and Engineering, 290:496--523, 2015.Google ScholarCross Ref
- L. C. McInnes, B. Smith, H. Zhang, and R. T. Mills. Hierarchical Krylov and nested Krylov methods for extreme-scale computing. Parallel Computing, 40(1):17--31, 2014. Google ScholarDigital Library
- E. H. Müller and R. Scheichl. Massively parallel solvers for elliptic partial differential equations in numerical weather and climate prediction. Quarterly Journal of the Royal Meteorological Society, 140(685):2608--2624, 2014.Google ScholarCross Ref
- S. Reiter, A. Vogel, I. Heppner, M. Rupp, and G. Wittum. A massively parallel geometric multigrid solver on hierarchically distributed grids. Computing and Visualization in Science, 16(4):151--164, 2013. Google ScholarDigital Library
- G. Rokos and G. Gorman. PRAgMaTIc--parallel anisotropic adaptive mesh toolkit. In R. Keller, D. Kramer, and J.-P. Weiss, editors, Facing the Multicore-Challenge III, volume 7686 of Lecture Notes in Computer Science, pages 143--144. Springer Berlin Heidelberg, 2013.Google Scholar
- J. Rudi, A. C. I. Malossi, T. Isaac, G. Stadler, M. Gurnis, P. W. Staar, Y. Ineichen, C. Bekas, A. Curioni, and O. Ghattas. An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in Earth's mantle. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, page 5. ACM, 2015. Google ScholarDigital Library
- K. Rupp, F. Rudolf, and J. Weinbub. ViennaCL - A high level linear algebra library for GPUs and multicore CPUs. In International Workshop on GPUs and Scientific Applications, pages 51--56, 2010.Google Scholar
- J. Shewchuk. Triangle: A two-dimensional quality mesh generator and Delaunay triangulator. http://www-2.cs.cmu.edu/~quake/triangle.html, 2005.Google Scholar
- J. R. Shewchuk. Triangle: Engineering a 2D quality mesh generator and Delaunay triangulator. In M. C. Lin and D. Manocha, editors, Applied Computational Geometry: Towards Geometric Engineering, volume 1148 of Lecture Notes in Computer Science, pages 203--222. Springer-Verlag, May 1996. From the 1st ACM Workshop on Applied Computational Geometry. Google ScholarDigital Library
- H. Si. TetGen: A quality tetrahedral mesh generator and three-dimensional Delaunay triangulator. http://tetgen.berlios.de, 2005.Google Scholar
- H. Si. TetGen, a Delaunay-based quality tetrahedral mesh generator. ACM Transactions on Mathematical Software (TOMS), 41(2), February 2015. Google ScholarDigital Library
- H. Sundar, G. Biros, C. Burstedde, J. Rudi, O. Ghattas, and G. Stadler. Parallel geometric-algebraic multigrid on unstructured forests of octrees. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, page 43. IEEE Computer Society Press, 2012. Google ScholarDigital Library
- U. Trottenberg, C. W. Oosterlee, and A. Schuller. Multigrid. Academic Press, 2000. Google ScholarDigital Library
- Extreme-Scale Multigrid Components within PETSc
Recommendations
Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in Hypre and PETSc
* Copper Mountain Special Issue on Iterative MethodsWe describe our software package Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) recently publicly released. BLOPEX is available as a stand-alone serial library, as an external package to PETSc (Portable, Extensible Toolkit for ...
An Evaluation of Parallel Multigrid as a Solver and a Preconditioner for Singularly Perturbed Problems
* Special Issue on Iterative Methods for Solving Systems of Algebraic EquationsIn this paper we try to achieve h-independent convergence with preconditioned GMRES ([Y. Saad and M. H. Schultz, SIAM J. Sci. Comput., 7 (1986), pp. 856--869]) and BiCGSTAB ([H. A. Van der Vorst, SIAM J. Sci. Comput., 13 (1992), pp. 63--644]) for two-...
Multigrid Methods for Implicit Runge--Kutta and Boundary Value Method Discretizations of Parabolic PDEs
Advanced time discretization schemes for stiff systems of ordinary differential equations (ODEs), such as implicit Runge--Kutta and boundary value methods, have many appealing properties. However, the resulting systems of equations can be quite large and ...
Comments