Jia Guo
ABSTRACT
Today's mobile devices are equipped with cameras capable of taking very high-resolution pictures. For computer vision tasks which require relatively low resolution, such as image classification, sub-sampling is desired to reduce the unnecessary power consumption of the image sensor. In this paper, we study the relationship between subsampling and the performance degradation of image classifiers that are based on deep neural networks (DNNs). We empirically show that subsampling with the same step size leads to very similar accuracy changes for different classifiers. In particular, we could achieve over 15x energy savings just by subsampling while suffering almost no accuracy lost. For even better energy accuracy trade-offs, we propose AdaSkip, where the row sampling resolution is adaptively changed based on the image gradient. We implement AdaSkip on an FPGA and report its energy consumption.
- 2018. iPhone X - Technical Specifications. https://www.apple.com/iphone-x/specs/. (2018). Accessed: 2018-03-01.Google Scholar
- 2018. Xperia™XZ Specifications - Sony Mobile. https://www.sonymobile.com/us/products/phones/xperia-xz/specifications/. (2018). Accessed: 2018-03-01.Google Scholar
- Evgeny Artyomov and Orly Yadid-Pecht. 2006. Adaptive Multiple-Resolution CMOS Active Pixel Sensor. IEEE Trans. Circuits Syst. 53-1, 10 (2006), 2178--2186.Google ScholarCross Ref
- Mark Buckler, Suren Jayasuriya, and Adrian Sampson. 2017. Reconfiguring the Imaging Pipeline for Computer Vision. In ICCV. 975--984.Google Scholar
- Huaijin G. Chen, Suren Jayasuriya, Jiyue Yang, Judy Stephen, Sriram Sivara-makrishnan, Ashok Veeraraghavan, and Alyosha C. Molnar. 2016. ASP Vision: Optically Computing the First Layer of Convolutional Neural Networks Using Angle Sensitive Pixels. In CVPR. 903--912.Google Scholar
- Jeng-Hau Lin et al. 2017. Binarized Convolutional Neural Networks with Separable Filters for Efficient Hardware Acceleration. In CVPR Workshops. 344--352.Google Scholar
- Kawahito Shoji et al. 1997. A CMOS image sensor with analog two-dimensional DCT-based compression circuits for one-chip cameras. J. Solid-State Circuits 32, 12 (1997), 2030--2041.Google ScholarCross Ref
- Robert LiKamWa et al. 2013. Energy characterization and optimization of image sensing toward continuous mobile vision. In MobiSys. 69--82. Google ScholarDigital Library
- Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In ICLR.Google Scholar
- Hongxiang Gu and Viswanathan Swaminathan. 2018. From Thumbnails to Summaries - A single Deep Neural Network to Rule Them All. In ICME. To appear.Google Scholar
- Jia Guo and Miodrag Potkonjak. 2017. Pruning ConvNets Online for Efficient Specialist Models. In CVPR Workshops. 430--437.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770--778.Google Scholar
- Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Q. Weinberger. 2018. Multi-Scale Dense Convolutional Networks for Efficient Prediction. In ICLR.Google Scholar
- Sabrina E. Kemeny, Roger Panicacci, Bedabrata Pain, Larry H. Matthies, and Eric R. Fossum. 1997. Multiresolution image sensor. IEEE Trans. Circuits Syst. Video Techn. 7, 4 (1997), 575--583. Google ScholarDigital Library
- Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Master's thesis. Department of Computer Science, University of Toronto.Google Scholar
- Ian Kuon and Jonathan Rose. 2007. Measuring the Gap Between FPGAs and ASICs. IEEE Trans, on CAD of Integrated Circuits and Systems 26, 2 (2007), 203--215. Google ScholarDigital Library
- Walter D. Leon-Salas, Sina Balkir, Khalid Sayood, Michael W. Hoffman, and Nathan Schemm. 2006. A CMOS imager with focal plane compression. In ISCAS.Google Scholar
- Robert LiKamWa, Yunhui Hou, Yuan Gao, Mia Polansky, and Lin Zhong. 2016. RedEye: Analog ConvNet Image Sensor Architecture for Continuous Mobile Vision. In ISCA. 255--266. Google ScholarDigital Library
- Yusuke Oike and Abbas El Gamal. 2013. CMOS Image Sensor With Per-Column ΣΔ ADC and Programmable Compressed Sensing. J. Solid-State Circuits 48, 1 (2013), 318--328.Google ScholarCross Ref
- Olga Russakovsky and et al. 2015. ImageNet Large Scale Visual Recognition Challenge. Int. J. Computer Vision 115, 3 (2015), 211--252. Google ScholarDigital Library
Index Terms
- Efficient Image Sensor Subsampling for DNN-Based Image Classification
Recommendations
Energy characterization and optimization of image sensing toward continuous mobile vision
MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and servicesA major hurdle to frequently performing mobile computer vision tasks is the high power consumption of image sensing. In this work, we report the first publicly known experimental and analytical characterization of CMOS image sensors. We find that modern ...
Subsampling-Adaptive Directional Wavelet Transform for Image Coding
DCC '10: Proceedings of the 2010 Data Compression ConferenceIn lifting-based directional wavelet transforms, different subsampling patterns may show significant difference for directional signals in image coding. This paper investigates the influence of subsampling in directional wavelet transform. We show that ...
Energy proportional image sensors for continuous mobile vision
MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and servicesA hurdle to frequently performing mobile computer vision tasks is the high energy cost of image sensing. In particular, modern image sensors are not energy proportional; for low resolution and low frame rate capture, the image sensor consumes almost the ...
Comments