|
 ABSTRACT
This paper describes a novel spatial-force decomposition for N-body simulations for which we observe O(sqrt(p)) communication scaling. This has enabled Blue Matter to approach the effective limits of concurrency for molecular dynamics using particle-mesh (FFT-based) methods for handling electrostatic interactions. Using this decomposition, Blue Matter running on Blue Gene/L has achieved simulation rates in excess of 1000 time steps per second and demonstrated significant speed-ups to O(1) atom per node. Blue Matter employs a Communicating Sequential Process (CSP) style model with application communication state machines compiled to hardware interfaces. The scalability achieved has enabled methodologically rigorous biomolecular simulations on biologically interesting systems, such as membrane-bound proteins, whose time scales dwarf previous work on those systems, Major scaling improvements will require exploration of alternative algorithms for treating the long range electrostatics.
REFERENCES
Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.
| |
1
|
F. Allen , G. Almasi , W. Andreoni , D. Beece , B. J. Berne , A. Bright , J. Brunheroto , C. Cascaval , J. Castanos , P. Coteus , P. Crumley , A. Curioni , M. Denneau , W. Donath , M. Eleftheriou , B. Fitch , B. Fleischer , C. J. Georgiou , R. Germain , M. Giampapa , D. Gresh , M. Gupta , R. Haring , H. Ho , P. Hochschild , S. Hummel , T. Jonas , D. Lieber , G. Martyna , K. Maturu , J. Moreira , D. Newns , M. Newton , R. Philhower , T. Picunko , J. Pitera , M. Pitman , R. Rand , A. Royyuru , V. Salapura , A. Sanomiya , R. Shah , Y. Sham , S. Singh , M. Snir , F. Suits , R. Swetz , W. C. Swope , N. Vishnumurthy , T. J. C. Ward , H. Warren , R. Zhou, Blue Gene: a vision for protein science using a petaflop supercomputer, IBM Systems Journal, v.40 n.2, p.310-327, February 2001
|
| |
2
|
Deserno, M., and Holm, C. 1998. How to mesh up Ewald sums. i. a theoretical and numerical comparison of various particle mesh routines. J. Chem. Phys. 109, 18, 7678--7693.
|
| |
3
|
Eleftheriou, M., Fitch, B., Rayshubskiy, A., Ward, T., and Germain, R. 2005. Performance measurements of the 3d FFT on the Blue Gene/L supercomputer. In Euro-Par 2005 Parallel Processing: 11th International Euro-Par Conference, Lisbon, Portugal, August 30-September 2, 2005. Springer-Verlag, J. Cunha and P. Medeiros, Eds., vol. 3648 of Lecture Notes in Computer Science, 795--803.
|
| |
4
|
Eleftheriou, M., Germain, R., Royyuru, A., and Zhou, R. 2006. Thermal denaturing of mutant lysozyme with both oplsaa and charmm force fields. submitted to J. Am. Chem. Soc.
|
| |
5
|
Eleftheriou, M., Rayshubskiy, A., Pitera, J. W., Fitch, B. G., Zhou, R., and Germain, R. S. 2006. Parallel implementation of the replica exchange molecular dynamics algorithm on Blue Gene/L. In Fifth IEEE International Workshop on High Performance Computational Biology.
|
| |
6
|
B. G. Fitch , R. S. Germain , M. Mendell , J. Pitera , M. Pitman , A. Rayshubskiy , Y. Sham , F. Suits , W. Swope , T. J. C. Ward , Y. Zhestkov , R. Zhou, Blue Matter, an application framework for molecular simulation on blue gene, Journal of Parallel and Distributed Computing, v.63 n.7-8, p.759-773, July 2003
[doi> 10.1016/S0743-7315(03)00084-4]
|
| |
7
|
Fitch, B. G., Rayshubskiy, A., Eleftheriou, M., Ward, T. C., Giampapa, M., Zhestkov, Y., Pitman, M. C., Suits, F., Grossfield, A., Pitera, J., Swope, W., Zhou, R., Germain, R. S., and Feller, S. 2005. Blue matter: Strong scaling of molecular dynamics on Blue Gene/L. Research Report RC23688, IBM Research Division, August.
|
| |
8
|
Fitch, B. G., Rayshubskiy, A., Eleftheriou, M., Ward, T. C., Giampapa, M., Zhestkov, Y., Pitman, M. C., Suits, F., Grossfield, A., Pitera, J., Swope, W., Zhou, R., Feller, S., and Germain, R. S. 2006. Blue Matter: Strong scaling of molecular dynamics on Blue Gene/L. In International Conference on Computational Science (ICCS 2006), Springer-Verlag, V. Alexandrov, D. van Albada, P. Sloot, and J. Dongarra, Eds., vol. 3992 of LNCS, 846--854.
|
| |
9
|
|
| |
10
|
Gara, A., ET AL. 2005. Overview of the Blue Gene/L system architecture. IBM Journal of Research and Development 49, 2/3, 195--212.
|
 |
11
|
Robert S. Germain , Blake Fitch , Aleksandr Rayshubskiy , Maria Eleftheriou , Michael C. Pitman , Frank Suits , Mark Giampapa , T.J. Christopher Ward, Blue matter on blue gene/L: massively parallel computation for biomolecular simulation, Proceedings of the 3rd IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis, September 19-21, 2005, Jersey City, NJ, USA
[doi> 10.1145/1084834.1084888]
|
| |
12
|
Germain, R., Zhestkov, Y., Eleftheriou, M., Rayshubskiy, A., Suits, F., Ward, T., and Fitch, B. 2005. Early performance data on the Blue Matter molecular simulation framework. IBM Journal of Research and Development 49, 2/3, 447--456.
|
| |
13
|
Giampapa, M., Bellofatto, R., Blumrich, M. A., Chen, D., Dombrowa, M. B., Gara, A., Haring, R. A., Heidelberger, P., Hoenicke, D., Kopcsay, G. V., Nathanson, B. J., Steinmacher-Burow, B. D., Ohmacht, M., Salapura, V., and Vranas, P. 2005, Blue Gene/L advanced diagnostics environment. IBM Journal of Research and Development 49, 2/3, 319--332.
|
| |
14
|
Grossfield, A., Feller, S. E., and Pitman, M. C. 2006. A role for direct interactions in the modulation of rhodopsin by omega-3 polyunsaturated lipids. PNAS 103, 13, 4888--4893.
|
| |
15
|
|
| |
16
|
|
| |
17
|
Kumar, S., Huang, C., Almasi, G., and Kale, L. V. 2006. Achieving strong scaling with NAMD on Blue Gene/l. 20th IEEE International Parallel and Distributed Processing Symposium, IEEE. http://charm.cs.uiuc.edu/papers/NAMDIDPDSO6.pdf.
|
| |
18
|
James C. Phillips , Gengbin Zheng , Sameer Kumar , Laxmikant V. Kalé, NAMD: biomolecular simulation on thousands of processors, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-18, November 16, 2002, Baltimore, Maryland
|
| |
19
|
Pitman, M. C., Suits, F., MacKerell, Alexander D., J., and Feller, S. E. 2004. Molecular-level organization of saturated and polyunsaturated fatty acids in a phosphatidylcholine bilayer containing cholesterol. Biochemistry 43, 49, 15318--15328.
|
| |
20
|
Pitman, M. C., Grossfield, A., Suits, F., and Feller, S. E. 2005. Role of cholesterol and polyunsaturated chains in lipid-protein interactions: Molecular dynamics simulation of rhodopsin in a realistic membrane environment. Journal of the American Chemical Society 127, 13, 4576--4577.
|
| |
21
|
Pitman, M. C., Suits, F., Gawrisch, K., and Feller, S. E. 2005. Molecular dynamics investigation of dynamical properties of phosphatidylethanolamine lipid bilayers. Journal of Chemical Physics 122, 24, 244715.
|
| |
22
|
Plimpton, S., and Hendrickson, B. 1996. A new parallel method for molecular dynamics simulation of macromolecular systems. Journal of Computational Chemistry 17, 3, 326--337.
|
| |
23
|
Shaw, D. 2004. An asymptotic improvement in the parallel evaluation of pairwise particle interactions. Presented at Philadelpha American Chemical Society meeting, September.
|
| |
24
|
Shaw, D. E. 2005. A fast, scalable method for the parallel evaluation of distance-limited pairwise particle interactions. Journal of Computational Chemistry 26, 13, 1318--1328.
|
| |
25
|
Snir, M. 2004. A note on n-body computations with cutoffs. Theory of Computing Systems 37, 295--318. DOI: 10.1007/s00224-003-1071-0.
|
| |
26
|
Sugita, Y., and Okamoto, Y. 1999. Replica-exchange molecular dynamics method for protein folding. Chem. Phys. Lett. 314, 141--151.
|
| |
27
|
Suits, F., Pitman, M. C., and Feller, S. E. 2005. Molecular dynamics investigation of the structural properties of phosphatidylethanolamine lipid bilayers. Journal of Chemical Physics 122, 24.
|
| |
28
|
Swope, W., Andersen, H., Berens, P., and Wilson, K. 1982. A computer simulation method for the calculation of equilibrium constants for the formation of physical clusters of molecules: Application to small water clusters. Journal of Chemical Physics 76, 637--649.
|
| |
29
|
Swope, W. C., Pitera, J. W., Suits, F., Pitman, M., Eleftheriou, M., Fitch, B. G., Germain, R. S., Rayshubskiy, A., Ward, T. J. C., Zhestkov, Y., and Zhou, R. 2004. Describing protein folding kinetics by molecular dynamics simulations. 2. example applications to alanine dipeptide and a β-hairpin peptide. J. Phys. Chem. B 108, 21, 6582--6594.
|
| |
30
|
|
CITED BY 5
|
|
José E. Moreira , Valentina Salapura , George Almasi , Charles Archer , Ralph Bellofatto , Peter Bergner , Randy Bickford , Mathias Blumrich , José R. Brunheroto , Arthur A. Bright , Michael Brutman , José G. Castaños , Dong Chen , Paul Coteus , Paul Crumley , Sam Ellis , Thomas Engelsiepen , Alan Gara , Mark Giampapa , Tom Gooding , Shawn Hall , Ruud A. Haring , Roger Haskin , Philip Heidelberger , Dirk Hoenicke , Todd Inglett , Gerrard V. Kopcsay , Derek Lieber , David Limpert , Pat McCarthy , Mark Megerian , Mike Mundy , Martin Ohmacht , Jeff Parker , Rick A. Rand , Don Reed , Ramendra Sahoo , Alda Sanomiya , Richard Shok , Brian Smith , Gordon G. Stewart , Todd Takken , Pavlos Vranas , Brian Wallenfelt , Michael Blocksome , Joe Ratterman, The blue gene/L supercomputer: a hardware and software story, International Journal of Parallel Programming, v.35 n.3, p.181-206, June 2007
|
|
|
S. Kumar , C. Huang , G. Zheng , E. Bohm , A. Bhatele , J. C. Phillips , H. Yu , L. V. Kalé, Scalable molecular dynamics with NAMD on the IBM Blue Gene/L system, IBM Journal of Research and Development, v.52 n.1, p.177-188, January 2008
|
|
|
David E. Shaw , Martin M. Deneroff , Ron O. Dror , Jeffrey S. Kuskin , Richard H. Larson , John K. Salmon , Cliff Young , Brannon Batson , Kevin J. Bowers , Jack C. Chao , Michael P. Eastwood , Joseph Gagliardo , J. P. Grossman , C. Richard Ho , Douglas J. Ierardi , István Kolossváry , John L. Klepeis , Timothy Layman , Christine McLeavey , Mark A. Moraes , Rolf Mueller , Edward C. Priest , Yibing Shan , Jochen Spengler , Michael Theobald , Brian Towles , Stanley C. Wang, Anton, a special-purpose machine for molecular dynamics simulation, ACM SIGARCH Computer Architecture News, v.35 n.2, May 2007
|
|
|
B. G. Fitch , A. Rayshubskiy , M. Eleftheriou , T. J. C. Ward , M. E. Giampapa , M. C. Pitman , J. W. Pitera , W. C. Swope , R. S. Germain, Blue matter: scaling of N-body simulations to one atom per node, IBM Journal of Research and Development, v.52 n.1, p.145-158, January 2008
|
|
|
|
|
Adobe Acrobat
QuickTime
Windows Media Player
Real Player
Html
G.1