BIM-Tracker: A model-based visual tracking approach for indoor localisation using a 3D building model

Abstract

This article presents an accurate and robust visual indoor localisation approach that not only is infrastructure-free, but also avoids accumulation error by taking advantage of (1) the widespread ubiquity of mobile devices with cameras and (2) the availability of 3D building models for most modern buildings. Localisation is performed by matching image sequences captured by a camera, with a 3D model of the building in a model-based visual tracking framework. Comprehensive evaluation of the approach with a photo-realistic synthetic dataset shows the robustness of the localisation approach under challenging conditions. Additionally, the approach is tested and evaluated on real data captured by a smartphone. The results of the experiments indicate that a localisation accuracy better than 10 cm can be achieved by using this approach. Since localisation errors do not accumulate the proposed approach is suitable for indoor localisation tasks for long periods of time and augmented reality applications, without requiring any local infrastructure. A MATLAB implementation can be found on https://github.com/debaditya-unimelb/BIM-Tracker.

Introduction

Indoor location information is the key enabler of a range of applications including navigation guidance, location-based services, emergency response, guiding vulnerable people and augmented reality. Indoor environments present a challenge for localisation due to strong attenuation of Global Navigation Satellite System (GNSS) signals (Mautz, 2012) compared to the outdoor environments. In the literature, many different approaches for indoor localisation have been proposed. However, the performance of indoor localisation is still lagging behind compared to outdoor localisation (Lymberopoulos and Liu, 2017), and many localisation applications are waiting for an acceptable solution.

Present indoor localisation approaches are either infrastructure-dependent, whose installation and maintenance are costly and not always feasible, or are infrastructure-free, which are not accurate enough for mass-market applications (Alarifi et al., 2016, Mautz, 2012). By infrastructure, we mean a dedicated network of sensors, transmitters or beacons installed in the indoor environment. Consequently, infrastructure-free indoor localisation has become a focus of research and development during the past decade, and improvements in indoor localisation systems are likely to generate better prospects for business. Among various infrastructure-free approaches, those based on digital television signals, FM radio signals, magnetic field, ambient sound levels and barometers provide metre level accuracy which is not sufficient for many indoor location-based applications (Xie et al., 2014, Muralidharan et al., 2014, Ye et al., 2014, Tarzia et al., 2011, Serant et al., 2011).

Other infrastructure-free methods such as pedestrian dead reckoning (PDR), visual odometry, and simultaneous localisation and mapping (SLAM), suffer from the accumulation of localisation errors resulting in the drift of the estimated trajectory (Khoshelham and Ramezani, 2017, Scaramuzza and Fraundorfer, 2011, Caron et al., 2014). The reported accuracy of the PDR using inertial measurement units (IMU) combined with recalibration from other sources such as Wi-Fi is approximately 1 m (Lymberopoulos and Liu, 2017) or 1% of the trajectory length (Mautz, 2012). A drift of 1% is acceptable for short distances but is not suitable for long distance indoor localisation applications. Moreover, visual odometry and SLAM methods are susceptible to poorly textured indoor environments such as a corridor, due to the lack of image features. Additionally, SLAM involves performing a loop closure, which is not practical for navigation applications, e.g. in a long tunnel, as the user cannot be forced to make loops.

Model-based visual tracking methods overcome the above challenges by using a 3D model to recalibrate the drift (Lepetit and Fua, 2005). Furthermore, the methods using model-based visual tracking (such as the work of Drummond and Cipolla, 2002) eliminate the requirement of textured indoor environments, and are computationally inexpensive (Lepetit and Fua, 2005). However, these model-based tracking approaches have been designed for tracking small objects and in small spaces such as a room and are therefore unsuitable for continuous localisation in large indoor environments. Moreover, the existing works that use model-based visual tracking lacks a comprehensive evaluation of the achievable accuracy, robustness and the estimated trajectory.

In this paper we present BIM-Tracker: a model-based tracking approach to indoor localisation that is based on matching images captured by a mobile device with the corresponding view of a building information model (BIM). The edges in the images are matched with the edges derived from the BIM to estimate the location of the camera in the BIM coordinate system by model-based visual tracking. The advantage of performing localisation in a BIM coordinate system is that the estimated locations are not prone to drifts compared to the incremental tracking methods that perform localisation by local motion estimation (Khoshelham and Ramezani, 2017). Because most of the smartphones and smartglasses are equipped with a camera, and the fact that a low level-of-detail 3D model of the environment is usually available or can be easily generated, the present research proposes a model-based visual tracking approach for accurate and drift-free localisation in an infrastructure-free indoor environment. The following are the main contributions of the article:

• We formulate an MSAC (M-estimator sample consensus) framework to use two hypotheses on either side of a back-projected model line to search the corresponding image edges. This strategy is a balance between the higher computational costs of using several multiple hypotheses for tracking and the robustness that it provides.

• We provide experimental insight into the optimal camera configurations and factors that contribute to errors for a model-based visual tracking approach in an indoor environment. A detailed analysis of the estimated trajectory is performed using a photo-realistic synthetic dataset with several configurations such as different image resolutions, camera field-of-views (FOV), motion blur, clutter and occlusions.

• We demonstrate the ability of BIM-Tracker for drift-free localisation using real images, which makes it suitable for navigation and augmented reality applications.¹

The paper proceeds with a review of visual methods and related works in the field of model-based tracking in Section 2. The theory and methodology for model-based visual tracking using edges are explained in Section 3. The experimental design and the evaluation results are discussed in Section 4, followed by conclusions in Section 6.