Abstract
Deep Neural Networks emerged in the last years as the most promising approach to the smart processing of data. However, their effectiveness is still a challenge when they are implemented in resource-constrained architectures, such as those of edge devices often requiring at least the inference phase. This work investigates the impact of two different weight compression techniques initially designed and tested for DNN hardware accelerators in a scenario involving general-purpose low-end hardware. After applying several levels of weight compression on the MobileNet DNN model, we show how accelerator-oriented weight compression techniques can positively impact both memory traffic pressure and inference/latency figures, resulting in some cases in a good trade-off in terms of accuracy loss.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Fifth version of the Hierarchical Data Format (HDF) a file format designed to store and organize large amounts of data.
- 3.
psrecord is available online at: https://github.com/astrofrog/psrecord.
References
Agustsson, E., et al.: Soft-to-hard vector quantization for end-to-end learning compressible representations. In: International Conference on Neural Information Processing Systems, pp. 1141–1151 (2017)
Ascia, G., Catania, V., Mineo, A., Monteleone, S., Palesi, M., Patti, D.: Improving inference latency and energy of dnns through wireless enabled multi-chip-module-based architectures and model parameters compression. In: 2020 14th IEEE/ACM International Symposium on Networks-on-Chip (NOCS), pp. 1–6. IEEE (2020)
Cavigelli, L., Rutishauser, G., Benini, L.: Ebpc: Extended bit-plane compression for deep neural network inference and training accelerators. IEEE J. Emerg. Sel. Top. Circ. Syst. 9(4), 723–734 (2019)
Cheng, Y., Wang, D., Zhou, P., Zhang, T.: A survey of model compression and acceleration for deep neural networks (2017)
Choi, Y., El-Khamy, M., Lee, J.: Towards the limit of network quantization. CoRR abs/1612.01543 (2016)
Gudovskiy, D.A., Hodgkinson, A., Rigazio, L.: DNN feature map compression using learned representation over GF(2). In: European Conference on Computer Vision, September 2018
Han, S., Mao, H., Dally, W.: Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In: International Conference on Learning Representations, October 2016
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2, 1097–1105 (2012)
Lahdhiri, H., et al.: Dnnzip: selective layers compression technique in deep neural network accelerators. In: 2020 23rd Euromicro Conference on Digital System Design (DSD), pp. 526–533. IEEE (2020)
Li, E., Zeng, L., Zhou, Z., Chen, X.: Edge AI: on-demand accelerating deep neural network inference via edge computing. IEEE Trans. Wireless Commun. 19(1), 447–457 (2019)
Mazumder, A.N., et al.: A survey on the optimization of neural network accelerators for micro-AI on-device inference. IEEE J. Emerg. Sel. Top. Circ. Syst. 11(4), 532–547 (2021)
Russo, E., Palesi, M., Monteleone, S., Patti, D., Mineo, A., Ascia, G., Catania, V.: DNN model compression for IoT domain specific hardware accelerators. IEEE Internet Things J. 9(9), 6650–6662 (2022)
Wu, C.J., et al.: Machine learning at Facebook: understanding inference at the edge. In: 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 331–344. IEEE (2019)
Acknowledgements
This work was supported by the Research Grants from: the Italian Ministry of University and Research (MIUR) PNR 2015 2020 within the project “MAIA Monitoraggio Attivo dell Infrastruttura”, ref. ARS01_0035; MIUR PO FESR SICILIA 2014–2020 within the project “PRE-CUBE: PREdizione, PREvenzione, PREdisposizione”, ref. 086201000304; the University of Catania - Piaceri 2020–2022 - Linea 2, within the project MANGO.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Canzonieri, G., Monteleone, S., Palesi, M., Russo, E., Patti, D. (2022). Analyzing the Impact of DNN Hardware Accelerators-Oriented Compression Techniques on General-Purpose Low-End Boards. In: Awan, I., Younas, M., Poniszewska-Marańda, A. (eds) Mobile Web and Intelligent Information Systems. MobiWIS 2022. Lecture Notes in Computer Science, vol 13475. Springer, Cham. https://doi.org/10.1007/978-3-031-14391-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-14391-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14390-8
Online ISBN: 978-3-031-14391-5
eBook Packages: Computer ScienceComputer Science (R0)