Skip to main content
SpringerLink
Log in

Multiple Binary OxRAMs as Synapses for Convolutional Neural Networks

  • Chapter
  • First Online:
Advances in Neuromorphic Hardware Exploiting Emerging Nanoscale Devices

Part of the book series: Cognitive Systems Monographs ((COSMOS,volume 31))

Abstract

Oxide-based resistive memory (OxRAM) devices find applications in memory, logic, and neuromorphic computing systems. Among the different dielectrics proposed in OxRAM stacks, hafnium oxide, HfO\(_{2}\), attracted growing interest because of its compatibility with typical BEOL advanced CMOS processing and promising performances in terms of endurance (higher than Flash) and switching speed (few tens of ns). This chapter describes an artificial synapse composed of multiple binary HfO\(_{2}\)-based OxRAM cells connected in parallel, thereby providing synaptic analog behavior. The VRRAM technology is presented as a possible solution to gain area with respect to planar approaches by realizing one VRRAM pillar per synapse. The HfO\(_{2}\)-based OxRAM synapse has been proposed for hardware implementation of power efficient Convolutional Neural Networks for visual pattern recognition applications. Finally, the synaptic weight resolution and the robustness to device variability of the network have been investigated. Statistical evaluation of device variability is obtained on a 16 kbit OxRAM memory array integrated into advanced 28 nm CMOS technology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mead, C.: Neuromorphic electronic systems. Proc. IEEE 78, 1629–1636 (1990)

    Article  Google Scholar 

  2. Indiveri, G.: Shih-Chii Liu: memory and information processing in neuromorphic systems. Proc. IEEE 103, 1379–1397 (2015)

    Article  Google Scholar 

  3. Prezioso, M., Merrikh-Bayat, F., Hoskins, B.D., Adam, G.C., Likharev, K.K., Strukov, D.B.: Training and operation of anintegrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64 (2015)

    Article  Google Scholar 

  4. Burr, G.W., Shelby, R.M., di Nolfo, C., Jang, J.W., Shenoy, R.S., Narayanan, P., Virwani, K., Giacometti, E.U., Kurdi, B., Hwang, H.: Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element. In: Electron Devices Meeting (IEDM) IEEE International, pp. 29.5.1–29.5.4 (2014)

    Google Scholar 

  5. Kuzum, D., Yu, S., Wong, H.-S.P.: Synaptic electronics: materials, devices and applications. Nanotechnology 24, 382001 (2013)

    Article  Google Scholar 

  6. Suri, M., Bichler, O., Querlioz, D., Palma, G., Vianello, E., Vuillaume, D., Gamrat, C.: DeSalvo: CBRAM devices as binary synapses for lowpower stochastic neuromorphic systems: auditory (Cochlea) and visual (Retina) cognitive processing applications. In: Electron Devices Meeting (IEDM) IEEE International, pp. 10.3.1–10.3.4 (2012)

    Google Scholar 

  7. Kuzum, D., Jeyasingh, R.G.D., Wong,H.-S.P.: Energy efficient programming of nanoelectronic synaptic devices for large-scale implementation of associative and temporal sequence learning. In: Electron Devices Meeting (IEDM) IEEE International, pp. 30.3.1–30.3.4 (2011)

    Google Scholar 

  8. Suri, M., Bichler, O., Querlioz, D., Cueto, O., Perniola, L., Sousa, V., Vuillaume, D., Gamrat, C., DeSalvo, B.: Phase change memory as synapse for ultra-dense neuromorphic systems: application to complex visual pattern extraction. In: Electron Devices Meeting (IEDM) IEEE International, pp. 4.4.1–4.4.4 (2011)

    Google Scholar 

  9. Wang, I.-T., Lin, Y.-C., Wang, Y.-F., Hsu, C.-W., Hou, T.-H.: 3D synaptic architecture with ultralow sub-10 fJ energy per spike for neuromorphic computation. In: Electron Devices Meeting (IEDM) IEEE International, pp. 28.5.1–28.5.4 (2014)

    Google Scholar 

  10. Park, S., Sheri, A., Kim, J., Noh, J., Jang, J., Jeon, M., Lee, B., Lee, B.R., Lee, B.H., Hwang, H.: Neuromorphic speech systems using advanced ReRAM-based synapse. In: Electron Devices Meeting (IEDM) IEEE International, pp. 25.6.1–25.6.4 (2013)

    Google Scholar 

  11. Vianello, E., Thomas, O., Molas, G., Turkyilmaz, O., Jovanovic, N., Garbin, D., Palma, G., Alayan, M., Nguyen, C., Coignus, J., Giraud, B., Benoist, T., Reyboz, M., Toffoli, A., Charpin, C., Clermidy, F., Perniola, L.: Resistive memories for ultra-low-power embedded computing design. In: Electron Devices Meeting (IEDM) IEEE International, pp. 6.3.1–6.3.4 (2014)

    Google Scholar 

  12. Benoist, A., Blonkowski, S., Jeannot, S., Denorme, S., Damiens, J., Berger, J., Candelier, P., Vianello, E., Grampeix, H., Nodin, J.E., Jalaguier, E., Perniola, L., Allard, B.: 28nm advanced CMOS resistive RAM solution as embedded non-volatile memory. In: Proceedings of the IEEE Reliability Physics Symposium, pp. 2E.6.1–2E.6.5 (2014)

    Google Scholar 

  13. Govoreanu, B., Kar, G.S., Chen, Y., Paraschiv, V., Kubicek, S., Fantini, A., Radu, I.P., Goux, L., Clima, S., Degraeve, R., Jossart, N., Richard, O., Vandeweyer, T., Seo, K., Hendrickx, P., Pourtois, G., Bender, H., Altimime, L., Wouters, D.J., Kittl, J.A., Jurczak, M.: 10x10nm\(^2\) Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation. In: Electron Devices Meeting (IEDM) IEEE International, pp. 31.6.1–31.6.4 (2011)

    Google Scholar 

  14. Garbin, D., Suri, M., Bichler, O., Querlioz, D., Gamrat, C., DeSalvo, B.: Probabilistic neuromorphic system using binary phase-change memory (PCM) synapses: detailed power consumption analysis. In: 3th IEEE Conference on Nanotechnology (IEEE-NANO), pp. 91–94 (2013)

    Google Scholar 

  15. Garbin, D., Vianello, E., Bichler, O., Rafhay, Q., Gamrat, C., Ghibaudo, G., DeSalvo, B., Perniola, L.: HfO\(_2\)-based OxRAM devices as synapses for convolutional neural networks. IEEE Trans. Electron Devices 62, 2494–2501 (2015)

    Article  Google Scholar 

  16. Bill, J., Legenstein, R.: A compound memristive synapse model for statistical learning through STDP in spiking neural networks. Front. Neurosci. 8, 412 (2014)

    Google Scholar 

  17. Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: Seventh IEEE International Conference on Document Analysis and Recognition, pp. 958–963 (2003)

    Google Scholar 

  18. Goldberg, D.H., Cauwenberghs, G., Andreou, A.G.: Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-and-fire neurons. Neural Netw. 14, 781–793 (2001)

    Article  Google Scholar 

  19. Piccolboni, G., Molas, G., Portal, J.M., Coquand, R., Bocquet, M., Garbin, D., Vianello, E., Carabasse, C., Delaye, V., Pellissier, C., Magis, T., Cagli, C., Gely, M., Cueto, O., Deleruyelle, O., Ghibaudo, G., DeSalvo, B., Perniola, L.: Investigation of the potentialities of Vertical Resistive RAM (VRRAM) for neuromorphic applications. In: lectron Devices Meeting (IEDM) IEEE International, pp. 17.2.1–17.2.4 (2015)

    Google Scholar 

  20. Garbin, D., Bichler, O., Vianello, E., Rafhay, Q., Gamrat, C., Perniola, L., Ghibaudo, G., DeSalvo, B.: Variability-tolerant convolutional neural network for pattern recognition applications based on OxRAM synapses. In: Electron Devices Meeting (IEDM) IEEE International, pp. 28.4.1–28.4.4 (2014)

    Google Scholar 

  21. Kim, S., Ishii, M., Lewis, S., Perri, T., BrightSky, M., Kim, W., Jordan, R., Burr, G.W., Sosa, N., Ray, A., Han, J.-P., Miller, C., Hosokawa, K., Lam, C.: NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with On-Chip neuron circuits for continuous in-situ learning. In: Electron Devices Meeting (IEDM) IEEE International, pp. 17.1.1–17.1.4 (2015)

    Google Scholar 

  22. Wang, Z., Ambrogio, S., Baletti, S., Ielemini, D.: A 2-transistor/1-resistor artificial synapse capable of communication and stochastic learning in neuromorphic systems. Front. Neurosci. 8, 438 (2015)

    Article  Google Scholar 

  23. Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cats visual cortex. J. Physiol. 160, 106–154 (1962)

    Article  Google Scholar 

  24. Felleman, D.J., Van Essen, D.C.: Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991)

    Article  Google Scholar 

  25. Fukushima, K.: Artificial vision by multi-layered neural networks: neocognitron and its advances. Neural Netw. 37, 103–119 (2013)

    Article  Google Scholar 

  26. Ciresan, D., Meier, U., Masci, J., Schmidhuber, J.: Multi-column deep neural network for traffic sign classification. Neural Netw. 32, 333–338 (2012)

    Article  Google Scholar 

  27. Ning, F., Delhomme, D., LeCun, Y., Piano, F., Bottou, L., Barbano, P.E.: Toward automatic phenotyping of developing embryos from videos. IEEE Trans. Image Process. 14, 1360–1371 (2005)

    Article  Google Scholar 

  28. Sermanet, P., Kavukcuoglu, K., Chintala, S., LeCun, Y.: Pedestrian detection with unsupervised multi-stage feature learning. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 36263633 (2013)

    Google Scholar 

  29. Vaillant, R., Monrocq, C., LeCun, Y.: A convolutional neural network hand tracker. IEEE Proc.-Vision, Image. Signal Process. 141, 245–250 (1994)

    Google Scholar 

  30. Nowlan, S.J., Platt, J.C.: Original approach for the localisation of objects in images. Adv. Neural Inf. Process. Syst. 901–908 (1995)

    Google Scholar 

  31. Garcia, C., Delakis, M.: Convolutional face finder: a neural architecture for fast and robust face detection. IEEE Trans. Pattern Anal. Mach. Intell. 26, 14081423 (2004)

    Article  Google Scholar 

  32. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1701–1708 (2014)

    Google Scholar 

  33. Garbin, D., Vianello, E., Bichler, O., Azzaz, M., Rafhay, Q., Candelier, P., Gamrat, C., Ghibaudo, G., DeSalvo, B., Perniola, L.: On the impact of OxRAM-based synapses variability on convolutional neural networks performance. In: 2015 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH), pp. 193–198 (2015)

    Google Scholar 

  34. LeCun, Y., Cortes, C., Burges, C.J.C.: The MNIST Database of Handwritten Digits. http://yann.lecun.com/exdb/mnist/

  35. Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: The German traffic sign recognition benchmark: a multi-class classification competition. In: Proceedings of the IEEE International Joint Conference on Neural Networks, pp. 1453–1460 (2011)

    Google Scholar 

  36. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)

    Article  Google Scholar 

  37. Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649 (2012)

    Google Scholar 

  38. Bichler, O., Querlioz, D., Thorpe, S. J., Bourgoin, J.-P., Gamrat, C.: Unsupervised features extraction from asynchronous silicon retina through spike-timing-dependent plasticity. In: Proceedings of the IEEE International Joint Conference on Neural Networks, pp. 859–866 (2011)

    Google Scholar 

  39. Hayakawa, Y., Himeno, A., Yasuhara, R., Boullart, W., Vecchio, E., Vandeweyer, T., Witters, T., Crotti, D., Jurczak, M., Fujii, S., Ito, S., Kawashima, Y., Ikeda, Y., Kawahara, A., Kawai, K., Wei, K., Muraoka, S., Shimakawa, K., Mikawa, T., Yoneda, S.: Highly reliable TaO\(_x\) ReRAM with centralized filament for 28 nm embedded application. In: 2015 Symposium on VLSI Technology Digest of Technical Papers (2015)

    Google Scholar 

  40. Ueki, M., Takeuchi, K., Yamamoto, T., Tanabe, A., Ikarashi, N., Saitoh, M., Nagumo, T., Sunamura, H., Narihiro, M., Uejima, K., Masuzaki, K., Furutake, N., Saito, S., Yabe, Y., Mitsuiki, A., Takeda, K., Hase, T., Hayashi, Y.: Low-Power embedded ReRAM technology for IoT applications. In: 2015 Symposium on VLSI Technology Digest of Technical Papers (2015)

    Google Scholar 

  41. Vianello, E., Garbin, D., Jovanovic, N., Bichler, O., Thomas, O., Salvo, B., Perniola, L.: Oxide based resistive memories for low power embedded applications and neuromorphic systems. In: 2015 ECS Transactions (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CEA LETI MINATEC Campus, 17 Rue des Martyrs, 38054, Grenoble Cedex 9, France

    E. Vianello, D. Garbin, G. Piccolboni, G. Molas, B. De Salvo & L. Perniola

  2. CEA LIST, 91191, Gif-sur-yvette, France

    O. Bichler

Authors
  1. E. Vianello
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. Garbin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. Bichler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. G. Piccolboni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. G. Molas
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. B. De Salvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. L. Perniola
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. Vianello .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, Indian Institute of Technology Delhi, New Delhi, Delhi, India

    Manan Suri

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer (India) Pvt. Ltd.

About this chapter

Cite this chapter

Vianello, E. et al. (2017). Multiple Binary OxRAMs as Synapses for Convolutional Neural Networks. In: Suri, M. (eds) Advances in Neuromorphic Hardware Exploiting Emerging Nanoscale Devices. Cognitive Systems Monographs, vol 31. Springer, New Delhi. https://doi.org/10.1007/978-81-322-3703-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-81-322-3703-7_6

  • Published:

  • Publisher Name: Springer, New Delhi

  • Print ISBN: 978-81-322-3701-3

  • Online ISBN: 978-81-322-3703-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation