Fig. 1. AR-kit components: a high end laptop and a custom stereo video see-through HMD are employed for visualization. An inertial sensor and a custom high speed CMOS camera are used for tracking (a). Snapshots: our custom stereo video see-through set consisting of Fire-i firewire webcams and an I-visor 4400VPD HMD (b), hybrid tracking unit consisting of custom CMOS camera and an XSens MT9 (c), the backpack with laptop for visualization (d), our custom SBC case (e).

Fig. 2. Vampire application scenario (HMD Screenshot): this example shows the localization of objects (book, cup), textual annotation (book), and several dialogs.

Fig. 3. VAMPIRE: this figure sketches the realization of our cognitive vision system on several computers. The AR subsystem is used to teach the active memory by pointing, initiating object recognition and plane decomposition tasks. Besides, it is used to start retrieval processes and to visualize results.

Fig. 4. 3D Cursor: to the left, the idea of the 3D cursor application is depicted. The tracking system processes the corner features of the CR target for self-localization. The selection of the phone with a 3D cursor allows to estimate the position of this object in scene coordinates (a). An artificial corner target [5] is used as an intermediate step on the way to natural features. The target is identified by the perspectively invariant cross ratios (CR) of the segments on the two intersecting lines. The pose can be calculated from the positions of the corners (b). The figure on the right sketches the relations described in Eq. 1 (c).

Fig. 5. Exploiting disparity: a horizontal displacement d of the cursor from the center of the image plane emulates depth in the stereo HMD (a). The user focuses an object which is perceived in the center of the image shown in the HMD, although there is actually a displacement in the images provided by the two cameras (b).

Fig. 6. Experiments and results: for calibration, a hyperbola fitting is performed (a) and its accuracy is compared to the standard stereo vision procedure (d). Three users (+) without any experience and one well trained user (*) placed the 3D cursor two times on the surface of an object (distance = 1,2, …, 5 m). It can be seen that the achieved accuracy depends on training and distance to the object (b and e). Up to a distance of 4 m, the error of the automated cursor is <2 cm for a static scene (c and f). It clearly outperforms the manual cursor, because the automated cursor uses a more reliable stereo correspondence provided by template matching.

Fig. 7. 3D Cursor in combination with an active camera: AR-kit and active camera use the same scene coordinate system defined by the artificial CR-target (a). After the estimation of the object’s position t_obj with the 3D cursor, the view of the active camera is directed to the object to record a different image of the object in order to obtain extra information (b).

Fig. 8. Evaluation setup: (a) Snapshot recorded during evaluation. (b) Object distribution layout.

Fig. 9. Cursor primitives.

Mobile AR-kit components

Request rates vs. window sizes (pixels) and number of windows (n): request denotes a cycle consisting of window positioning and read-out