skip to main content
10.1145/1094811.1094836acmconferencesArticle/Chapter ViewAbstractPublication PagessplashConference Proceedingsconference-collections
Article

Quantifying the performance of garbage collection vs. explicit memory management

Published:12 October 2005Publication History

ABSTRACT

Garbage collection yields numerous software engineering benefits, but its quantitative impact on performance remains elusive. One can compare the cost of conservative garbage collection to explicit memory management in C/C++ programs by linking in an appropriate collector. This kind of direct comparison is not possible for languages designed for garbage collection (e.g., Java), because programs in these languages naturally do not contain calls to free. Thus, the actual gap between the time and space performance of explicit memory management and precise, copying garbage collection remains unknown.We introduce a novel experimental methodology that lets us quantify the performance of precise garbage collection versus explicit memory management. Our system allows us to treat unaltered Java programs as if they used explicit memory management by relying on oracles to insert calls to free. These oracles are generated from profile information gathered in earlier application runs. By executing inside an architecturally-detailed simulator, this "oracular" memory manager eliminates the effects of consulting an oracle while measuring the costs of calling malloc and free. We evaluate two different oracles: a liveness-based oracle that aggressively frees objects immediately after their last use, and a reachability-based oracle that conservatively frees objects just after they are last reachable. These oracles span the range of possible placement of explicit deallocation calls.We compare explicit memory management to both copying and non-copying garbage collectors across a range of benchmarks using the oracular memory manager, and present real (non-simulated) runs that lend further validity to our results. These results quantify the time-space tradeoff of garbage collection: with five times as much memory, an Appel-style generational collector with a non-copying mature space matches the performance of reachability-based explicit memory management. With only three times as much memory, the collector runs on average 17% slower than explicit memory management. However, with only twice as much memory, garbage collection degrades performance by nearly 70%. When physical memory is scarce, paging causes garbage collection to run an order of magnitude slower than explicit memory management.

References

  1. Hardware --- G5 Performance Programming, Dec. 2003. Available at http://developer.apple.com/hardware/ve/g5.html.Google ScholarGoogle Scholar
  2. B. Alpern, C. R. Attanasio, J. J. Barton, M. G. Burke, P. Cheng, J.-D. Choi, A. Cocchi, S. J. Fink, D. Grove, M. Hind, S. F. Hummel, D. Lieber, V. Litvinov, M. F. Mergen, T. Ngo, V. Sarkar, M. J. Serrano, J. C. Shepherd, S. E. Smith, V. C. Sreedhar, H. Srinivasan, and J. Whaley. The Jalepeño virtual machine. IBM Systems Journal, 39(1), Feb. 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. B. Alpern, C. R. Attanasio, J. J. Barton, A. Cocchi, S. F. Hummel, D. Lieber, T. Ngo, M. Mergen, J. C. Shepherd, and S. Smith. Implementing Jalepeño in Java. In Proceedings of SIGPLAN 1999 Conference on Object-Oriented Programming, Languages, & Applications, volume 34(10) of ACM SIGPLAN Notices, pages 314--324, Denver, CO, Oct. 1999. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A. W. Appel. Garbage collection can be faster than stack allocation. Information Processing Letters, 25(4):275--279, 1987. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. A. W. Appel. Allocation without locking. Software Practice and Experience, 19(7), 1989. Short Communication. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. E. D. Beger. The hoard memory allocator. Available at http://www.hoard.org.Google ScholarGoogle Scholar
  7. E. D. Berger, B. G. Zorn, and K. S. McKinley. Composing high-performance memory allocators. In Proceedings of the 2001 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Snowbird, Utah, June 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. E. D. Berger, B. G. Zorn, and K. S. McKinley. Reconsidering custom memory allocation. In OOPSLA'02 ACM Conference on Object-Oriented Systems, Languages and Applications, ACM SIGPLAN Notices, Seattle, WA, Nov. 2002. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. E. D. Berger, B. G. Zorn, and K. S. McKinley. Reconsidering custom memory allocation. In Proceedings of the Conference on Object-Oriented Programming: Systems, Languages, and Applications (OOPSLA) 2002, Seattle, Washington, Nov. 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. M. Blackburn, P. Cheng, and K. S. McKinley. Myths and reality: The performance impact of garbage collection. In SIGMETRICS - Performance 2004, Joint International Conference on Measurement and Modeling of Computer Systems, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. M. Blackburn, P. Cheng, and K. S. McKinley. Oil and Water? High Performance Garbage Collection in Java with MMTk. In ICSE 2004, 26th International Conference on Software Engineering, page to appear, May 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. S. M. Blackburn and K. S. McKinley. Ulterior reference counting: Fast garbage collection without a long wait. In OOPSLA 2003 ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, Oct. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. H.-J. Boehm. Reducing garbage collector cache misses. In T. Hosking, editor, ISMM 2000 Proceedings of the Second International Symposium on Memory Management, volume 36(1) of ACM SIGPLAN Notices, Minneapolis, MN, Oct. 2000. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. H.-J. Boehm and M. Weiser. Garbage collection in an uncooperative environment. Software: Practice and Experience, 18(9):807--820, Sept. 1988. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. D. Burger, T. M. Austin, and S. Bennett. Evaluating future microprocessors: The SimpleScalar tool set. Computer Sciences Technical Report CS-TR-1996-1308, University of Wisconsin-Madison, Madison, WI, 1996.Google ScholarGoogle Scholar
  16. G. Colvin, B. Dawes, and D. Adler. C++ Boost Smart Pointers, Oct. 2004. Available at http://www.boost.org/libs/smart_ptr/smart_ptr.htm.Google ScholarGoogle Scholar
  17. S. P. E. Corporation. Specjbb2000. Available at http://www.spec.org/jbb2000/docs/userguide.html.Google ScholarGoogle Scholar
  18. S. P. E. Corporation. Specjvm98 documentation, Mar. 1999.Google ScholarGoogle Scholar
  19. D. L. Detlefs. Concurrent garbage collection for C++. In P. Lee, editor, Topics in Advanced Language Implementation. MIT Press, 1991.Google ScholarGoogle Scholar
  20. A. Diwan, D. Tarditi, and E. Moss. Memory system performance of programs with intensive heap allocation. ACM Trans. Comput. Syst., 13(3):244--273, Aug. 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. Dolby. Automatic inline allocation of objects. In Proceedings of SIGPLAN'97 Conference on Programming Languages Design and Implementation, ACM SIGPLAN Notices, Las Vegas, Nevada, June 1997. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Dolby and Chien. An evaluation of object inline allocation techniques. In OOPSLA'98 ACM Conference on Object-Oriented Systems, Languages and Applications, ACM SIGPLAN Notices, Vancouver, Oct. 1998. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. J. Dolby and Chien. An automatic object inlining and its evaluation. In Proceedings of SIGPLAN 2000 Conference on Programming Languages Design and Implementation, ACM SIGPLAN Notices, pages 345--357, Vancouver, June 2000. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Y. Feng and E. D. Berger. A locality-improving dynamic memory allocator. In Proceedings of the ACM SIGPLAN 2005 Workshop on Memory System Performance (MSP), Chicago, Illinois, June 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. R. R. Fenichel and J. C. Yochelson. A LISP garbage-collector for virtual-memory computer systems. Commun. ACM, 12(11):611--612, Nov. 1969. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. D. Gay and A. Aiken. Memory management with explicit regions. In Proceedings of the 1998 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 313 -- 323, Montreal, Canada, June 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. D. Gay and A. Aiken. Language support for regions. In Proceedings of the 2001 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 70 -- 80, Snowbird, Utah, June 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. W. Gloger. Dynamic memory allocator implementations in linux system libraries. Available at http://www.dent.med.uni-muenchen.de/~wmglo/malloc-slides.html.Google ScholarGoogle Scholar
  29. D. Grossman, G. Morrisett, T. Jim, M. Hicks, Y. Wang, and J. Cheney. Region-based memory management in cyclone. In Proceedings of the 2002 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 282--293, Berlin, Germany, June 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. J. Guy L Steele. Multiprocessing compactifying garbage collection. Communications of the ACM, 18(9):495--508, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. D. R. Hanson. Fast allocation and deallocation of memory based on object lifetimes. In Software Practice & Experience, volume 20(1), pages 5--12. Wiley, Jan. 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. M. Hertz and E. D. Berger. Garbage collection without paging. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 143--153, June 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. M. Hertz, S. M. Blackburn, J. E. B. Moss, K. S. McKinley, and D. Stefanović. Error-free garbage collection traces: How to cheat and not get caught. In Proceedings of the International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 2002), pages 140--151, Marina Del Ray, CA, June 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. M. Hertz, N. Immerman, and J. E. B. Moss. Framework for analyzing garbage collection. In Foundations of Information Technology in the Era of Network and Mobile Computing: IFIP 17th World Computer Congress - TC1 Stream (TCS 2002), pages 230--241, Montreal, Canada, Aug. 2002. Kluwer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. M. Hicks, G. Morrisett, D. Grossman, and T. Jim. Experience with safe manual memory-management in Cyclone. In A. Diwan, editor, ISMM'04 Proceedings of the Third International Symposium on Memory Management, ACM SIGPLAN Notices, Vancouver, Oct. 2004. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. M. W. Hicks, J. T. Moore, and S. M. Nettles. The measured cost of copying garbage collection mechanisms. In Proceedings of the ACM SIGPLAN Conference on Functional Programming, pages 292--305. ACM, June 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Hirzel, A. Diwan, and T. Hosking. On the usefulness of liveness for garbage collection and leak detection. In J. L. Knudsen, editor, Proceedings of 15th European Conference on Object-Oriented Programming, ECOOP 2001, volume 2072 of Springer-Verlag, Budapest, June 2001. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. X. Huang, S. M. Blackburn, K. S. McKinley, J. E. B. Moss, Z. Wang, and P. Cheng. The garbage collection advantage: Improving program locality. In Proceeding of the ACM Conference on Object-Oriented Systems, Languages and Applications, Vancouver, BC, Canada, Oct. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. X. Huang, J. E. B. Moss, K. S. Mckinley, S. Blackburn, and D. Burger. Dynamic SimpleScalar: Simulating Java virtual machines. Technical Report TR-03-03, University of Texas at Austin, Feb. 2003.Google ScholarGoogle Scholar
  40. M. S. Johnstone and P. R. Wilson. The memory fragmentation problem: Solved? In International Symposium on Memory Management, Vancouver, B.C., Canada, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. R. E. Jones and R. Lins. Garbage Collection: Algorithms for Automatic Dynamic Memory Management. Wiley, Chichester, July 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. P.-H. Kamp. Malloc(3) revisited. http://phk.freebsd.dk/pubs/malloc.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. D. G. Korn and K.-P. Vo. In search of a better malloc. In USENIX Conference Proceedings, Summer 1985, pages 489--506, Portland, OR, 1985.Google ScholarGoogle Scholar
  44. D. Lea. A memory allocator, 1998. Available at http://g.oswego.edu/dl/html/malloc.html.Google ScholarGoogle Scholar
  45. J. McCarthy. Recursive functions of symbolic expressions and their computation by machine. Communications of the ACM, 3:184--195, 1960. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. M. Michael. Scalable lock-free dynamic memory allocation. In Proceedings of SIGPLAN 2004 Conference on Programming Languages Design and Implementation, ACM SIGPLAN Notices, Washington, DC, June 2004. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. J. S. Miller and G. J. Rozas. Garbage collection is fast, but a stack is faster. Technical Report AIM-1462, MIT AI Laboratory, Mar. 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. G. Phipps. Comparing observed bug and productivity rates for Java and C++. Software Practice & Experience, 29(4):345--358, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. N. Sachindran, J. E. B. Moss, and E. D. Berger. MC2: High-performance garbage collection for memory-constrained environments. In Proceedings of the ACM Conference on Object-Oriented Systems, Languages and Applications, Vancouver, BC, Canada, Oct. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. P. Savola. Lbnl traceroute heap corruption vulnerability. Available at http://www.securityfocus.com/bid/1739.Google ScholarGoogle Scholar
  51. R. Shaham, E. Kolodner, and M. Sagiv. Estimating the impact of liveness information on space consumption in Java. In D. Detlefs, editor, ISMM'02 Proceedings of the Third International Symposium on Memory Management, ACM SIGPLAN Notices, pages 64--75, Berlin, June 2002. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Y. Smaragdakis, S. F. Kaplan, and P. R. Wilson. The EELRU adaptive replacement algorithm. Performance Evaluation, 53(2):93--123, July 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. D. Sugalski. Squawks of the parrot: What the heck is: Garbage collection, June 2003. Available at http://www.sidhe.org/~dan/blog/archives/000200.html.Google ScholarGoogle Scholar
  54. M. Tofte and J.-P. Talpin. Region-based memory management. Information and Computation, 132(2):109--176, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. L. Torvalds. Re: Faster compilation speed, 2002. Available at http://gcc.gnu.org/ml/gcc/2002-08/msg00544.html.Google ScholarGoogle Scholar
  56. D. Ungar. Generation scavenging: A non-disruptive high performance storage reclamation algorithm. In Proceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, pages 157--167, Pittsburgh, Pennsylvania, Apr. 1984. ACM SIGPLAN Notices 19, 5 (May 1984). Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. B. Venners. Inside the Java Virtual Machine. McGraw-Hill Osborne Media, Jan. 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Wikipedia. Comparison of Java to C++, 2004. Available at http://en.wikipedia.org/wiki/Comparison_of_Java_to_Cplusplus.Google ScholarGoogle Scholar
  59. P. R. Wilson. Uniprocessor garbage collection techniques. In Y. Bekkers and J. Cohen, editors, Proceedings of International Workshop on Memory Management, volume 637 of Lecture Notes in Computer Science, St Malo, France, 16--18 Sept. 1992. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. P. R. Wilson, M. S. Lam, and T. G. Moher. Caching considerations for generational garbage collection. In Conference Record of the 1992 ACM Symposium on Lisp and Functional Programming, pages 32--42, San Francisco, CA, June 1992. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. T. Yang, M. Hertz, E. D. Berger, S. F. Kaplan, and J. E. B. Moss. Automatic heap sizing: Taking real memory into account. In Proceedings of the 2004 ACM SIGPLAN International Symposium on Memory Management, Nov. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. B. G. Zorn. The measured cost of conservative garbage collection. Software Practice and Experience, 23(7):733--756, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Quantifying the performance of garbage collection vs. explicit memory management

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      OOPSLA '05: Proceedings of the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications
      October 2005
      562 pages
      ISBN:1595930310
      DOI:10.1145/1094811
      • cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 40, Issue 10
        Proceedings of the 20th annual ACM SIGPLAN conference on Object oriented programming systems languages and applications
        October 2005
        531 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/1103845
        Issue’s Table of Contents

      Copyright © 2005 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 12 October 2005

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • Article

      Acceptance Rates

      Overall Acceptance Rate268of1,244submissions,22%

      Upcoming Conference

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader