Fitting line projections in non-central catadioptric cameras with revolution symmetry
Introduction
In central systems the projection surface of a 3D line is a plane passing through the viewpoint of the camera. In this class of projection some of the information of the 3D line is lost because any 3D line lying on this plane is projected on the same line-image. In other words, a 3D line occludes any other line located behind because the projection surface is a plane.
By contrast, in non-central systems the projection rays do not intersect a common viewpoint. The locus of the viewpoint is, in general, tangent to a caustic (Agrawal et al., 2010) which is an envelope surface of the projection rays. In particular, when the non-central system is axial and has symmetry of revolution the projection rays intersects the axis of symmetry (see Fig. 1) and the projection surface of a 3D line is composed by skew lines forming a ruled surface. If we consider a set of 4 skew projection rays, being generic lines1, there exist only two lines intersecting the given set (Teller and Hohmeyer, 1999): the original 3D line and, if the system is axially symmetric, the axis of revolution. This means that no additional line can intersect the set of projection rays. Hence in axial non-central projections, it is not possible to occlude a 3D line with other line. Main consequence of this property is that the complete geometry of a 3D line is mapped on a single non-central image. Therefore, if the system is calibrated, the 3D line can be completely recovered from at least 4 line-image points or projection rays. On the contrary, image-points do not provide 3D information since points always occlude other 3D points along the projection ray.
This property implies a great geometric advantage of lines with respect to points in non-central cameras and generates new possibilities in line-features based 3D vision and entails promising applications in robotics and scene modelling. The richer information about the localization of the line in the space facilitates the tracking of the features even if the photometric information is not very discriminative. Notice also that line features usually represent boundaries of the scene that remain even in texture-less scenes. These advantages have a direct influence in robot pose estimation and Simultaneous Localization and Mapping(SLAM) where texture-less scenes can induce drift in pose estimation due to the lack of significant features.
Some previous approaches have tried to fit 3D lines from a single image in non-central catadioptric systems. The approach presented by Teller and Hohmeyer (1999) exploits the intersection operator between lines for defining a linear system from 4 rays computing the two incident lines. Since in non-central systems with symmetry of revolution one of these lines is the axis of revolution, this approach is used in Caglioti and Gasparini (2005); Caglioti et al. (2007a); Gasparini and Caglioti (2011) for estimating 3D lines from line projections of non-central catadioptric images and studying the degeneracies and singular configurations. In Lanman et al. (2006) the same approach is used with spherical catadioptric mirrors for 3D reconstruction. Work in Swaminathan et al. (2008) extends the approach from lines to planar curves. In Agrawal et al. (2010) the line-image for spherical catadioptric systems is indirectly shown as the epipolar curve. This epipolar curve represents in fact the projection of a 3D line (the epipolar ray of other system) an is represented through a second order line complex. Some simplifications have been used to improve the reconstructions by reducing the DOFs of the problem: considering horizontal lines (Chen, Cheng, Xiong, Basu, Zhang, 2011, Pinciroli, Bonarini, Matteucci, 2005), exploiting cross-ratio properties (Perdigoto and Araujo, 2012) or imposing additional constraints such as parallelism or perpendicularity (Bermudez-Cameo et al., 2014).
Line projections have been also used to estimate the calibration of non-central systems in a generalization of the plumb-line approach. For example in Caglioti et al. (2007b) they exploit that there exists less ambiguity when the system is off-axis with impressive results. However, that approach does not allow to obtain an analytical expression of the line projection. In Agrawal and Ramalingam (2013) they exploit particular geometric properties of spherical mirrors for computing extrinsic calibration parameters. As application, the pose of non-central catadioptric systems is estimated in an image sequence (Miraldo, Araujo, 2014, Miraldo, Araujo, Goncalves, 2015) using known 3D lines.
In Perdigoto and Araujo (2016) a method for estimating the mirror shape and extrinsic parameters for axial non-central catadioptric systems is presented. For the particular case of spherical mirrors quartic curves representing line projections are fitted. For this fitting they propose using the geometric distance by using a generic constrained optimization.
Automatically extracting the projection of a line in non-central images is a challenging task. In omnidirectional central systems the problem differs if the system is calibrated (Puig et al., 2012) or not. It has been recently solved for calibrated images (Bazin, Demonceaux, Vasseur, Kweon, 2010, Ying, Zha, 2005) and for uncalibrated axially symmetric images (Bermudez-Cameo, Lopez-Nicolas, Guerrero, 2015, Tardif, Sturm, Roy, 2006). In non-central images the difficulty of the extraction increases due to the high distortions induced on line-images, the elevated number of degrees of freedom involved in the extraction and the lack of effective baseline of current non-central systems. This problem has been successfully addressed for non-central circular panoramas in Bermudez-Cameo et al. (2017).
In this paper, we study the robust fitting of line projections in non-central catadioptric cameras with symmetry of revolution and present a method for automatic extraction of these line projections. Up to our knowledge, this is the first work addressing this problem in non-central catadioptric cameras. The proposal has been developed for conical catadioptric and spherical catadioptric systems. This procedure automatically segments the collection of edges corresponding to line-images. The complete 3D localization of a line is also recovered from the extraction in a single image even if the accuracy of the result is sensitive to noise. The contributions of this work are the following:
- •
A unified framework for describing line-images in non-central systems with revolution symmetry.
- •
Polynomial expressions of the line-images for conical and spherical catadioptric systems.
- •
A closed-form solution for computing the geometry of the mirror from 5 points lying on a line-image in conical catadioptric systems.
- •
Solutions based on polynomial roots for computing the Euclidean distance from point to line-image for conical and spherical catadioptric systems.
- •
A feature for measuring effective baseline in a set of rays in non-central systems.
- •
An algorithm for automatic line-image extraction in non-central images.
- •
Accuracy comparison of line reconstruction between catadioptric systems of similar sizes.
Preliminary results of this work have been presented in Bermudez-Cameo et al. (2014b) where we show the equation of the line-image and the metric distance for conical catadioptric systems. In this paper we extend the approach to spherical catadioptric systems developing the equation of the line-image and the metric distance. We also contribute with the measure of the effective baseline of a set of rays and the automatic line-image extraction procedure. The proposed approaches have been independently validated with a set of simulations. Then the complete pipeline has been tested with synthetic and real images (Fig. 2).
In Section 2 we introduce the required background of our proposal. In Section 3 we present our unified framework for describing rays and line projections in non-central systems with symmetry of revolution. In Section 4 we present the proposed description of the line-image for conical catadioptric system and we introduce the computing of the mirror geometry parameter from 5 points of the line-image. In Section 5 we show the polynomial description of the line-image for spherical catadioptric systems and we particularize the corresponding metric distance for this case. In Section 6 we analytically solve the metric distance from a point to the line-image. We particularize the corresponding metric distance for conical and spherical catadioptric systems. In Section 7 we present the algorithm for line-image extraction in non-central images including the proposed feature for measuring the effective baseline of a set of rays. In Section 8 we evaluate the method with synthetic and real images. Finally in Section 9 we present the conclusions.
Section snippets
Background
In this section, we introduce the relevant geometric concepts and notation used in this paper. In particular, we summarize the description used for lines, which is based on Grassmann–Cayley algebra (Kanatani, 2015), the transformations between systems of references, and the side operator between two lines.
Line projection in non-central systems
In this section, we present a unified framework for describing rays and line projections in non-central systems with symmetry of revolution.
Conical mirror systems
In conical catadioptric systems with the camera located in the axis of revolution of the mirror, the locus of viewpoint is a circle of radius Rc centred in the vertical axis at height Zc (Baker, Nayar, 2001, López-Nicolás, Sagüés, 2014). The locus of this circle, which depends on the distance Zm between the camera and the vertex of the mirror, and the aperture angle τ of the mirror, is
Spherical mirror systems
A spherical catadioptric system is a non-central system composed by a spherical mirror and a perspective camera. Due to the symmetry of the sphere there exist symmetry of revolution wherever the perspective camera is located. The system is characterized by the radius of the sphere Rs, the distance from the camera to the center of the sphere Zs, the intrinsic parameters of the perspective camera and the relative rotation of the camera with respect to the axis of revolution. This axis is defined
Algebraic and metric distances
When evaluating if a point belongs to a line-image for fitting the curve is necessary a function measuring the distance from a point to the line-image. The quality of the extracted line-image depends on this distance. In this Section, we present a qualitative comparison among distances and we propose different approaches for tackling this problem. Having the line-image Eq. (9) a measure of distance is the algebraic distance which is measured in
Robust line extraction in non-central systems
In this Section, we present a method for line-image extraction in non-central systems. Line extraction in non-central systems is an unsolved challenging task due to the difficulties of mapping the 3D of the line on a 2D space, the sensitivity to noise in fitting, and the elevate number of degrees of freedom involved. Up to our knowledge, this is the first work addressing this problem.
Assuming the system is calibrated, four points of the line-image are needed to define a line-image and its
Experimental evaluation
In this section, we present the experimental evaluation of the proposal. First, we show simulations to study the behaviour of the proposed method. Then we present selected examples of line-extraction in synthetic and real images. Experiments with synthetic images allow testing the complete algorithm knowing the ground truth and with absence of calibration errors. The extraction method is also tested in images taken with real catadioptric systems. Finally we present several examples of
Conclusions
In this paper, we have analyzed the geometry of line projections in non-central catadioptric systems focusing on conical and spherical catadioptric cameras. In both cases line-images are quartic polynomials, although in the conical case it can be simplified to a compact description of 6 homogeneous parameters. The underlying structure of line-images has been used for designing a line-image extraction method and to obtain the geometry of the mirror in the case of the conical catadioptric camera.
Acknowledgments
This work was supported by the Spanish projects DPI2015-65962-R (MINECO/FEDER, UE) and DPI2014-61792-EXP.
References (38)
- et al.
Motion estimation by decoupling rotation and translation in catadioptric vision
Comput. Vis. Image Underst.
(2010) - et al.
Exploiting line metric reconstruction from non-central circular panoramas
Pattern Recognit. Lett
(2017) - et al.
A 2-point algorithm for 3D reconstruction of horizontal lines from a single omni-directional image
Pattern Recognit. Lett.
(2011) - et al.
Planar pose estimation for general cameras using known 3d lines
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
(2014) - et al.
Robust detection of 3D scene horizontal and vertical lines in conical catadioptric sensors
Proceedings of the Sixth Workshop on Omnidirectional Vision
(2005) - et al.
USAC: a universal framework for random sample consensus
IEEE Trans. Pattern Anal. Mach. Intel.
(2013) - et al.
Single image calibration of multi-axial imaging systems
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2013) - et al.
Analytical forward projection for axial non-central dioptric and catadioptric cameras
Proceedings of the Eleventh European Conference on Computer Vision (ECCV)
(2010) - et al.
Beyond Alhazen’s problem: analytical projection model for non-central catadioptric cameras with quadric mirrors
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2011) - et al.
Single Viewpoint Catadioptric Cameras
(2001)