Real-Time RGBD Reconstruction Using Structural Constraint for Indoor AR

Abstract

RGBD-based 3D indoor scene reconstruction has been paid much attention due to the advantage of consumer depth camera. It is significant for many interactive application, especially in augmented reality. At present, the AR system mainly focus on the issue of the instabilities in the registration without any marker (i.e. error accumulation in camera pose estimate). Current methods generally consider isolate point cloud pairwise as the argument in the registration and ignore the prior correlation of geometric structures in the indoor scene. In our work, we focus on the issue and propose a novel, structural-based AR framework. Specifically, we use a two-pass scheme strategy to execute the system. The first pass tracks camera and analyze scene structure timely at video rate. We apply structural constraint to the iterative-closest-point algorithm and generate a new pose optimization strategy. We also incorporate the structure information into the global model integration and improve the reconstruction quality. Comparing with other state-of-the-art online reconstruction methods, our approach significantly reduces pose drift. The second pass simultaneously processing occlusion between virtual objects and real scene with the advent of prior structure analysis to improve the realism in AR.

This work was supported in part by the National Natural Science Foundation of China under Grant 61572054, in part by the Applied Basic Research Program of Qingdao under Grant 16-10-1-3-xx.