Joel Coburn
Adrian M. Caulfield
Laura M. Grupp
Rajesh K. Gupta
Steven Swanson
Skip Abstract Section
Abstract
Persistent, user-defined objects present an attractive abstraction for working with non-volatile program state. However, the slow speed of persistent storage (i.e., disk) has restricted their design and limited their performance. Fast, byte-addressable, non-volatile technologies, such as phase change memory, will remove this constraint and allow programmers to build high-performance, persistent data structures in non-volatile storage that is almost as fast as DRAM. Creating these data structures requires a system that is lightweight enough to expose the performance of the underlying memories but also ensures safety in the presence of application and system failures by avoiding familiar bugs such as dangling pointers, multiple free()s, and locking errors. In addition, the system must prevent new types of hard-to-find pointer safety bugs that only arise with persistent objects. These bugs are especially dangerous since any corruption they cause will be permanent.
We have implemented a lightweight, high-performance persistent object system called NV-heaps that provides transactional semantics while preventing these errors and providing a model for persistence that is easy to use and reason about. We implement search trees, hash tables, sparse graphs, and arrays using NV-heaps, BerkeleyDB, and Stasis. Our results show that NV-heap performance scales with thread count and that data structures implemented using NV-heaps out-perform BerkeleyDB and Stasis implementations by 32x and 244x, respectively, by avoiding the operating system and minimizing other software overheads. We also quantify the cost of enforcing the safety guarantees that NV-heaps provide and measure the costs of NV-heap primitive operations.
- B. Aker. Libmemcached. http://libmemcached.org/.Google Scholar
- P. Allenbach. Java card 3: Classic functionality gets a connectivity boost, March 2009. http://java.sun.com/developer/technicalArticles/javacard/javacard3/.Google Scholar
- T. Andrews and C. Harris. Combining language and database advances in an object-oriented development environment. In OOPSLA '87: Conference proceedings on Object-oriented programming systems, languages and applications, pages 430--440, New York, NY, USA, 1987. ACM. Google ScholarDigital Library
- M. P. Atkinson, L. Daynès, M. J. Jordan, T. Printezis, and S. Spence. An orthogonally persistent java. SIGMOD Rec., 25(4):68--75, 1996. Google ScholarDigital Library
- D. A. Bader and K. Madduri. Design and implementation of the hpcs graph analysis benchmark on symmetric multiprocessors. In HiPC 2005: Proc. 12th International Conference on High Performance Computing, pages 465--476, December 2005. Google ScholarDigital Library
- E. D. Berger, K. S. McKinley, R. D. Blumofe, and P. R. Wilson. Hoard: a scalable memory allocator for multithreaded applications. In ASPLOS-IX: Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, pages 117--128, New York, NY, USA, 2000. ACM. Google ScholarDigital Library
- R. Biswas and E. Ort. The java persistence api - a simpler programming model for entity persistence, May 2006. http://java.sun.com/developer/technicalArticles/J2EE/jpa/.Google Scholar
- C. Blundell, J. Devietti, E. C. Lewis, and M. M. K. Martin. Making the fast case common and the uncommon case simple in unbounded transactional memory. In ISCA '07: Proceedings of the 34th annual international symposium on Computer architecture, pages 24--34, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- M. J. Breitwisch. Phase change memory. Interconnect Technology Conference, 2008. IITC 2008. International, pages 219--221, June 2008.Google ScholarCross Ref
- P. Butterworth, A. Otis, and J. Stein. The gemstone object database management system. Commun. ACM, 34(10):64--77, 1991. Google ScholarDigital Library
- M. J. Carey, D. J. DeWitt, G. Graefe, D. M. Haight, J. E. Richardson, D. T. Schuh, E. J. Shekita, and S. L. Vandenberg. The exodus extensible dbms project: an overview. pages 474--499, 1990. Google ScholarDigital Library
- R. G. Cattell. Object Data Management: Object-Oriented and Extended. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1994. Google ScholarDigital Library
- A. M. Caulfield, J. Coburn, T. I. Mollov, A. De, A. Akel, J. He, A. Jagatheesan, R. K. Gupta, A. Snavely, and S. Swanson. Understanding the impact of emerging non-volatile memories on high-performance, io-intensive computing. In Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC '10, pages 1--11, Washington, DC, USA, 2010. IEEE Computer Society. Google ScholarDigital Library
- A. M. Caulfield, A. De, J. Coburn, T. I. Mollov, R. K. Gupta, and S. Swanson. Moneta: A high-performance storage array architecture for next-generation, non-volatile memories. In Proceedings of the 43nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 43, pages 385--395, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- S. Cho and H. Lee. Flip-n-write: a simple deterministic technique to improve pram write performance, energy and endurance. In Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 42, pages 347--357, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- S. Chu. Memcachedb. http://memcachedb.org/.Google Scholar
- J. Condit, E. B. Nightingale, C. Frost, E. Ipek, B. Lee, D. Burger, and D. Coetzee. Better i/o through byte-addressable, persistent memory. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, SOSP '09, pages 133--146, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- A. Dearle, G. N. C. Kirby, and R. Morrison. Orthogonal persistence revisited. In Proceedings of the Second international conference on Object databases, ICOODB'09, pages 1--22, Berlin, Heidelberg, 2010. Springer-Verlag. Google ScholarDigital Library
- G. Dhiman, R. Ayoub, and T. Rosing. Pdram: a hybrid pram and dram main memory system. In DAC '09: Proceedings of the 46th Annual Design Automation Conference, pages 664--469, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- B. Dieny, R. Sousa, G. Prenat, and U. Ebels. Spin-dependent phenomena and their implementation in spintronic devices. VLSI Technology, Systems and Applications, 2008. VLSI-TSA 2008. International Symposium on, pages 70--71, April 2008.Google ScholarCross Ref
- J. E. Gottschlich and D. A. Connors. Dracostm: a practical c++ approach to software transactional memory. In LCSD '07: Proceedings of the 2007 Symposium on Library-Centric Software Design, pages 52--66, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- L. Hammond, V. Wong, M. Chen, B. D. Carlstrom, J. D. Davis, B. Hertzberg, M. K. Prabhu, H. Wijaya, C. Kozyrakis, and K. Olukotun. Transactional memory coherence and consistency. In ISCA '04: Proceedings of the 31st annual international symposium on Computer architecture, page 102, Washington, DC, USA, 2004. IEEE Computer Society. Google ScholarDigital Library
- D. M. Hansen, D. R. Adams, and D. K. Gracio. In the trenches with objectstore. TAPOS, 5(4):201--207, 1999. Google ScholarDigital Library
- T. Harris and K. Fraser. Language support for lightweight transactions. In OOPSLA '03: Proceedings of the 18th annual ACM SIGPLAN conference on Object-oriented programing, systems, languages, and applications, pages 388--402, New York, NY, USA, 2003. ACM. Google ScholarDigital Library
- R. Haskin, Y. Malachi, W. Sawdon, and G. Chan. Recovery management in quicksilver. In SOSP '87: Proceedings of the eleventh ACM Symposium on Operating systems principles, pages 107--108, New York, NY, USA, 1987. ACM. Google ScholarDigital Library
- M. Herlihy, V. Luchangco, M. Moir, and W. N. Scherer, III. Software transactional memory for dynamic-sized data structures. In PODC '03: Proceedings of the twenty-second annual symposium on Principles of distributed computing, pages 92--101, New York, NY, USA, 2003. ACM. Google ScholarDigital Library
- M. Herlihy and J. E. B. Moss. Transactional memory: architectural support for lock-free data structures. In ISCA '93: Proceedings of the 20th annual international symposium on Computer architecture, pages 289--300, New York, NY, USA, 1993. ACM. Google ScholarDigital Library
- R. L. Hudson, B. Saha, A.-R. Adl-Tabatabai, and B. C. Hertzberg. Mcrt-malloc: a scalable transactional memory allocator. In ISMM '06: Proceedings of the 5th international symposium on Memory management, pages 74--83, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- International technology roadmap for semiconductors: Emerging research devices, 2009.Google Scholar
- H. V. Jagadish, D. F. Lieuwen, R. Rastogi, A. Silberschatz, and S. Sudarshan. Dalí: A high performance main memory storage manager. In VLDB '94: Proceedings of the 20th International Conference on Very Large Data Bases, pages 48--59, San Francisco, CA, USA, 1994. Morgan Kaufmann Publishers Inc. Google ScholarDigital Library
- Jedec standard: Low power double data rate 2 (lpddr2), March 2009.Google Scholar
- T. Kawahara, R. Takemura, K. Miura, J. Hayakawa, S. Ikeda, Y. M. Lee, R. Sasaki, Y. Goto, K. Ito, T. Meguro, F. Matsukura, H. Takahashi, H. Matsuoka, and H. Ohno. 2 mb spram (spin-transfer torque ram) with bit-by-bit bi-directional current write and parallelizing-direction current read. Solid-State Circuits, IEEE Journal of, 43(1):109--120, Jan. 2008.Google Scholar
- E. K. Kolodner and W. E. Weihl. Atomic incremental garbage collection and recovery for a large stable heap. In SIGMOD '93: Proceedings of the 1993 ACM SIGMOD international conference on Management of data, pages 177--186, New York, NY, USA, 1993. ACM. Google ScholarDigital Library
- C. Lamb, G. Landis, J. Orenstein, and D. Weinreb. The objectstore database system. Commun. ACM, 34(10):50--63, 1991. Google ScholarDigital Library
- B. C. Lee, E. Ipek, O. Mutlu, and D. Burger. Architecting phase change memory as a scalable dram alternative. In ISCA '09: Proceedings of the 36th annual international symposium on Computer architecture, pages 2--13, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- B. Liskov. Distributed programming in argus. Commun. ACM, 31(3):300--312, 1988. Google ScholarDigital Library
- B. Liskov, A. Adya, M. Castro, S. Ghemawat, R. Gruber, U. Maheshwari, A. C. Myers, M. Day, and L. Shrira. Safe and efficient sharing of persistent objects in thor. In SIGMOD '96: Proceedings of the 1996 ACM SIGMOD international conference on Management of data, pages 318--329, New York, NY, USA, 1996. ACM. Google ScholarDigital Library
- D. E. Lowell and P. M. Chen. Free transactions with rio vista. In SOSP '97: Proceedings of the sixteenth ACM symposium on Operating systems principles, pages 92--101, New York, NY, USA, 1997. ACM. Google ScholarDigital Library
- C.-K. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S. Wallace, V. J. Reddi, and K. Hazelwood. Pin: building customized program analysis tools with dynamic instrumentation. In PLDI '05: Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, pages 190--200, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
- V. J. Marathe, M. F. Spear, C. Heriot, A. Acharya, D. Eisenstat, W. N. Scherer III, and M. L. Scott. Lowering the overhead of software transactional memory. In ACM SIGPLAN Workshop on Transactional Computing. Jun 2006. Held in conjunction with PLDI 2006. Expanded version available as TR 893, Department of Computer Science, University of Rochester, March 2006.Google Scholar
- A. Marquez, J. N. Zigman, and S. M. Blackburn. Fast portable orthogonally persistent java. Softw. Pract. Exper., 30(4):449--479, 2000. Google ScholarDigital Library
- Memcached. http://memcached.org/.Google Scholar
- J. Metz. E-mail and appointment falsification analysis analysis of e-mail and appointment falsification on microsoft outlook/exchange. http://sourceforge.net/projects/libpff/files/.Google Scholar
- http://office.microsoft.com/en-us/outlook/ha010563001033.aspx.Google Scholar
- C. Mohan, D. Haderle, B. Lindsay, H. Pirahesh, and P. Schwarz. Aries: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. ACM Trans. Database Syst., 17(1):94--162, 1992. Google ScholarDigital Library
- Berkeley db. http://www.oracle.com/technology/products/\\berkeley-db/index.html.Google Scholar
- J. O'Toole, S. Nettles, and D. Gifford. Concurrent compacting garbage collection of a persistent heap. In SOSP '93: Proceedings of the fourteenth ACM symposium on Operating systems principles, pages 161--174, New York, NY, USA, 1993. ACM. Google ScholarDigital Library
- M. K. Qureshi, J. Karidis, M. Franceschini, V. Srinivasan, L. Lastras, and B. Abali. Enhancing lifetime and security of pcm-based main memory with start-gap wear leveling. In MICRO 42: Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture, pages 14--23, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- M. K. Qureshi, V. Srinivasan, and J. A. Rivers. Scalable high performance main memory system using phase-change memory technology. In Proceedings of the 36th annual international symposium on Computer architecture, ISCA '09, pages 24--33, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- B. Saha, A.-R. Adl-Tabatabai, R. L. Hudson, C. C. Minh, and B. Hertzberg. Mcrt-stm: a high performance software transactional memory system for a multi-core runtime. In PPoPP '06: Proceedings of the eleventh ACM SIGPLAN symposium on Principles and practice of parallel programming, pages 187--197, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- M. Satyanarayanan, H. H. Mashburn, P. Kumar, D. C. Steere, and J. J. Kistler. Lightweight recoverable virtual memory. In SOSP '93: Proceedings of the fourteenth ACM symposium on Operating systems principles, pages 146--160, New York, NY, USA, 1993. ACM. Google ScholarDigital Library
- W. N. Scherer, III and M. L. Scott. Advanced contention management for dynamic software transactional memory. In PODC '05: Proceedings of the twenty-fourth annual ACM symposium on Principles of distributed computing, pages 240--248, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
- R. Sears and E. Brewer. Stasis: flexible transactional storage. In OSDI '06: Proceedings of the 7th symposium on Operating systems design and implementation, pages 29--44, Berkeley, CA, USA, 2006. USENIX Association. Google ScholarDigital Library
- M. Seltzer and M. Olson. Libtp: Portable, modular transactions for unix. In Proceedings of the 1992 Winter Usenix, pages 9--25, 1992.Google Scholar
- J. S. Shapiro and J. Adams. Design evolution of the eros single-level store. In ATEC '02: Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference, pages 59--72, Berkeley, CA, USA, 2002. USENIX Association. Google ScholarDigital Library
- N. Shavit and D. Touitou. Software transactional memory. In PODC '95: Proceedings of the fourteenth annual ACM symposium on Principles of distributed computing, pages 204--213, New York, NY, USA, 1995. ACM. Google ScholarDigital Library
- V. Singhal, S. V. Kakkad, and P. R. Wilson. Texas: An efficient, portable persistent store. In POS, pages 11--33, 1992.Google Scholar
- F. G. Soltis. Inside the as/400. Twenty Ninth Street Press, 1996. Google ScholarDigital Library
- R. Takemura, T. Kawahara, K. Miura, J. Hayakawa, S. Ikeda, Y. Lee, R. Sasaki, Y. Goto, K. Ito, T. Meguro, F. Matsukura, H. Takahashi, H. Matsuoka, and H. Ohno. 2mb spram design: Bi-directional current write and parallelizing-direction current read schemes based on spin-transfer torque switching. Integrated Circuit Design and Technology, 2007. ICICDT '07. IEEE International Conference on, pages 1--4, 30 2007-June 1 2007.Google ScholarCross Ref
- H. Volos, A. J. Tack, and M. M. Swift. Mnemosyne: Lightweight persistent memory. In ASPLOS '11: Proceeding of the 16th international conference on Architectural support for programming languages and operating systems, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- S. J. White and D. J. DeWitt. Quickstore: a high performance mapped object store. In SIGMOD '94: Proceedings of the 1994 ACM SIGMOD international conference on Management of data, pages 395--406, New York, NY, USA, 1994. ACM. Google ScholarDigital Library
- P. Zhou, B. Zhao, J. Yang, and Y. Zhang. A durable and energy efficient main memory using phase change memory technology. In ISCA '09: Proceedings of the 36th annual international symposium on Computer architecture, pages 14--23, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
Index Terms
- NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories
Recommendations
NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories
ASPLOS XVI: Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systemsPersistent, user-defined objects present an attractive abstraction for working with non-volatile program state. However, the slow speed of persistent storage (i.e., disk) has restricted their design and limited their performance. Fast, byte-addressable, ...
NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories
ASPLOS '11Persistent, user-defined objects present an attractive abstraction for working with non-volatile program state. However, the slow speed of persistent storage (i.e., disk) has restricted their design and limited their performance. Fast, byte-addressable, ...
NVM duet: unified working memory and persistent store architecture
ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systemsEmerging non-volatile memory (NVM) technologies have gained a lot of attention recently. The byte-addressability and high density of NVM enable computer architects to build large-scale main memory systems. NVM has also been shown to be a promising ...
Comments