A 9.52 ms Latency, and Low-power Streaming Depth-estimation Processor with Shifter-based Pipelined Architecture for Smart Mobile Devices

Sungpill Choi; Kyuho Jason Lee; Youngwoo Kim; Hoi-Jun Yoo

doi:10.5573/JSTS.2020.20.3.255

주제분류

...

저널정보

대한전자공학회
JOURNAL OF SEMICONDUCTOR TECHNOLOGY AND SCIENCE 학술저널
Journal of Semiconductor Technology and Science Vol.20 No.3
2020.6 255 - 270 (16page)
DOI : 10.5573/JSTS.2020.20.3.255

저자정보

Sungpill Choi (Korea Advanced Institute of Science and Technology)
Kyuho Jason Lee (Ulsan National Institute of Science and Technology)
Youngwoo Kim (Korea Advanced Institute of Science and Technology)
Hoi-Jun Yoo (Korea Advanced Institute of Science and Technology)

이용수
내서재: 0

내서재에 추가
되었습니다. 내서재에서
삭제되었습니다.

초록·키워드

오류제보하기

The 3D hand gesture interface (HGI) for virtual reality and mixed reality on smart mobile devices is strongly dependent upon the robust depth-estimation with low latency and power consumption. However, the conventional depth-estimation hardware such as active depth sensors and stereo matching accelerators cannot realize the always-on and natural 3D HGI on mobile platform due to their large power consumption from active depth sensors and computations as well as the massive external memory bandwidth, respectively. To resolve the limit, we propose a depth-estimation processor that realizes the always-on and natural 3D HGI with algorithm and hardware co-optimization. The processor features: 1) shifter-based adaptive support weight aggregation that replaces complex floating-point operations with integer operations to reduce power and bandwidth by 92.2% and 69.1%; 2) line streaming 7-stage pipeline architecture with aggregation pipeline reordering optimization to realize 94% utilization and 43.9% memory reduction; and 3) shifting register-based pipeline buffer optimization to reduce 29.8% area. The proposed depth-estimation processor realizes a real-time 3D HGI with 9.52 ms of latency under QVGA stereo inputs. It achieves external memory bandwidth reduction to 18.93 MB/s with 15.56 mW power and 2.8 mm² area, which are 4.1x and 6.9x more efficient than state-of-the-arts [9, 10], respectively.

#Energy-efficient digital circuit #depth-estimation #stereo vision #low power #high throughput ASIC #image processing #memory-efficient design

참고문헌 (29)

참고문헌 신청

Larry Li, “Time-of-Flight Camera ? An Introduction”, May 2014, Texas Instrument, June 2016, http://www.ti.com/lit/wp/sloa190b/sloa190b.pdf Payne, et al., 7.6 A 512×424 CMOS 3D Time-of- Flight image sensor with multi-frequency photodemodulation up to 130MHz and 2GS/s ADC, International Solid-State Circuits Conference, pp. 134-135, Feb. 2014

Hololens Hardware detail. Available: https://docs.microsoft.com/en-us/windows/mixed-reality/hololenshardware-details Cyrus S. Bamji et al., “1Mpixel 65nm BSI 320MHz Demodulated TOF image sensor with 3.5mm global shutter pixels and analog binning”, International Solid-State Circuits Conference, pp. 94-95, Feb. 2018.

REAL3TM hardware documentation. Available: https://www.infineon.com/dgdl/Infineon-REAL3%20Image%20Sensor%20Family-PB-v01_00-EN.PDF?fileId=5546d462518ffd850151a0afc2302a58