Fitting line projections in non-central catadioptric cameras with revolution symmetry

https://doi.org/10.1016/j.cviu.2018.01.003Get rights and content

Highlights

  • We study the underlying geometry behind line projection in conical and spherical non-central catadioptric cameras.

  • The first automatic line-image extraction method for calibrated non-central catadioptric cameras.

  • The paper opens the possibility of obtaining 3D information from lines in a single non-central view.

  • We propose a new measure (“effective baseline”) of the 3D reconstruction quality of a 3D line from the minimum set of rays.

Abstract

Line-images in non-central cameras contain much richer information of the original 3D line than line projections in central cameras. The projection surface of a 3D line in most catadioptric non-central cameras is a ruled surface, encapsulating the complete information of the 3D line. The resulting line-image is a curve which contains the 4 degrees of freedom of the 3D line. That means a qualitative advantage with respect to the central case, although extracting this curve is quite difficult. In this paper, we focus on the analytical description of the line-images in non-central catadioptric systems with symmetry of revolution. As a direct application we present a method for automatic line-image extraction for conical and spherical calibrated catadioptric cameras. For designing this method we have analytically solved the metric distance from point to line-image for non-central catadioptric systems. We also propose a distance we call effective baseline measuring the quality of the reconstruction of a 3D line from the minimum number of rays. This measure is used to evaluate the different random attempts of a robust scheme allowing to reduce the number of trials in the process. The proposal is tested and evaluated in simulations and with both synthetic and real images.

Introduction

In central systems the projection surface of a 3D line is a plane passing through the viewpoint of the camera. In this class of projection some of the information of the 3D line is lost because any 3D line lying on this plane is projected on the same line-image. In other words, a 3D line occludes any other line located behind because the projection surface is a plane.

By contrast, in non-central systems the projection rays do not intersect a common viewpoint. The locus of the viewpoint is, in general, tangent to a caustic (Agrawal et al., 2010) which is an envelope surface of the projection rays. In particular, when the non-central system is axial and has symmetry of revolution the projection rays intersects the axis of symmetry (see Fig. 1) and the projection surface of a 3D line is composed by skew lines forming a ruled surface. If we consider a set of 4 skew projection rays, being generic lines1, there exist only two lines intersecting the given set (Teller and Hohmeyer, 1999): the original 3D line and, if the system is axially symmetric, the axis of revolution. This means that no additional line can intersect the set of projection rays. Hence in axial non-central projections, it is not possible to occlude a 3D line with other line. Main consequence of this property is that the complete geometry of a 3D line is mapped on a single non-central image. Therefore, if the system is calibrated, the 3D line can be completely recovered from at least 4 line-image points or projection rays. On the contrary, image-points do not provide 3D information since points always occlude other 3D points along the projection ray.

This property implies a great geometric advantage of lines with respect to points in non-central cameras and generates new possibilities in line-features based 3D vision and entails promising applications in robotics and scene modelling. The richer information about the localization of the line in the space facilitates the tracking of the features even if the photometric information is not very discriminative. Notice also that line features usually represent boundaries of the scene that remain even in texture-less scenes. These advantages have a direct influence in robot pose estimation and Simultaneous Localization and Mapping(SLAM) where texture-less scenes can induce drift in pose estimation due to the lack of significant features.

Some previous approaches have tried to fit 3D lines from a single image in non-central catadioptric systems. The approach presented by Teller and Hohmeyer (1999) exploits the intersection operator between lines for defining a linear system from 4 rays computing the two incident lines. Since in non-central systems with symmetry of revolution one of these lines is the axis of revolution, this approach is used in Caglioti and Gasparini (2005); Caglioti et al. (2007a); Gasparini and Caglioti (2011) for estimating 3D lines from line projections of non-central catadioptric images and studying the degeneracies and singular configurations. In Lanman et al. (2006) the same approach is used with spherical catadioptric mirrors for 3D reconstruction. Work in Swaminathan et al. (2008) extends the approach from lines to planar curves. In Agrawal et al. (2010) the line-image for spherical catadioptric systems is indirectly shown as the epipolar curve. This epipolar curve represents in fact the projection of a 3D line (the epipolar ray of other system) an is represented through a second order line complex. Some simplifications have been used to improve the reconstructions by reducing the DOFs of the problem: considering horizontal lines (Chen, Cheng, Xiong, Basu, Zhang, 2011, Pinciroli, Bonarini, Matteucci, 2005), exploiting cross-ratio properties (Perdigoto and Araujo, 2012) or imposing additional constraints such as parallelism or perpendicularity (Bermudez-Cameo et al., 2014).

Line projections have been also used to estimate the calibration of non-central systems in a generalization of the plumb-line approach. For example in Caglioti et al. (2007b) they exploit that there exists less ambiguity when the system is off-axis with impressive results. However, that approach does not allow to obtain an analytical expression of the line projection. In Agrawal and Ramalingam (2013) they exploit particular geometric properties of spherical mirrors for computing extrinsic calibration parameters. As application, the pose of non-central catadioptric systems is estimated in an image sequence (Miraldo, Araujo, 2014, Miraldo, Araujo, Goncalves, 2015) using known 3D lines.

In Perdigoto and Araujo (2016) a method for estimating the mirror shape and extrinsic parameters for axial non-central catadioptric systems is presented. For the particular case of spherical mirrors quartic curves representing line projections are fitted. For this fitting they propose using the geometric distance by using a generic constrained optimization.

Automatically extracting the projection of a line in non-central images is a challenging task. In omnidirectional central systems the problem differs if the system is calibrated (Puig et al., 2012) or not. It has been recently solved for calibrated images (Bazin, Demonceaux, Vasseur, Kweon, 2010, Ying, Zha, 2005) and for uncalibrated axially symmetric images (Bermudez-Cameo, Lopez-Nicolas, Guerrero, 2015, Tardif, Sturm, Roy, 2006). In non-central images the difficulty of the extraction increases due to the high distortions induced on line-images, the elevated number of degrees of freedom involved in the extraction and the lack of effective baseline of current non-central systems. This problem has been successfully addressed for non-central circular panoramas in Bermudez-Cameo et al. (2017).

In this paper, we study the robust fitting of line projections in non-central catadioptric cameras with symmetry of revolution and present a method for automatic extraction of these line projections. Up to our knowledge, this is the first work addressing this problem in non-central catadioptric cameras. The proposal has been developed for conical catadioptric and spherical catadioptric systems. This procedure automatically segments the collection of edges corresponding to line-images. The complete 3D localization of a line is also recovered from the extraction in a single image even if the accuracy of the result is sensitive to noise. The contributions of this work are the following:

  • A unified framework for describing line-images in non-central systems with revolution symmetry.

  • Polynomial expressions of the line-images for conical and spherical catadioptric systems.

  • A closed-form solution for computing the geometry of the mirror from 5 points lying on a line-image in conical catadioptric systems.

  • Solutions based on polynomial roots for computing the Euclidean distance from point to line-image for conical and spherical catadioptric systems.

  • A feature for measuring effective baseline in a set of rays in non-central systems.

  • An algorithm for automatic line-image extraction in non-central images.

  • Accuracy comparison of line reconstruction between catadioptric systems of similar sizes.

Preliminary results of this work have been presented in Bermudez-Cameo et al. (2014b) where we show the equation of the line-image and the metric distance for conical catadioptric systems. In this paper we extend the approach to spherical catadioptric systems developing the equation of the line-image and the metric distance. We also contribute with the measure of the effective baseline of a set of rays and the automatic line-image extraction procedure. The proposed approaches have been independently validated with a set of simulations. Then the complete pipeline has been tested with synthetic and real images (Fig. 2).

In Section 2 we introduce the required background of our proposal. In Section 3 we present our unified framework for describing rays and line projections in non-central systems with symmetry of revolution. In Section 4 we present the proposed description of the line-image for conical catadioptric system and we introduce the computing of the mirror geometry parameter from 5 points of the line-image. In Section 5 we show the polynomial description of the line-image for spherical catadioptric systems and we particularize the corresponding metric distance for this case. In Section 6 we analytically solve the metric distance from a point to the line-image. We particularize the corresponding metric distance for conical and spherical catadioptric systems. In Section 7 we present the algorithm for line-image extraction in non-central images including the proposed feature for measuring the effective baseline of a set of rays. In Section 8 we evaluate the method with synthetic and real images. Finally in Section 9 we present the conclusions.

Section snippets

Background

In this section, we introduce the relevant geometric concepts and notation used in this paper. In particular, we summarize the description used for lines, which is based on Grassmann–Cayley algebra (Kanatani, 2015), the transformations between systems of references, and the side operator between two lines.

Line projection in non-central systems

In this section, we present a unified framework for describing rays and line projections in non-central systems with symmetry of revolution.

Conical mirror systems

In conical catadioptric systems with the camera located in the axis of revolution of the mirror, the locus of viewpoint is a circle of radius Rc centred in the vertical axis at height Zc (Baker, Nayar, 2001, López-Nicolás, Sagüés, 2014). The locus of this circle, which depends on the distance Zm between the camera and the vertex of the mirror, and the aperture angle τ of the mirror, isRc=Zmsin2τ,Zc=Zm(1cos2τ).

Spherical mirror systems

A spherical catadioptric system is a non-central system composed by a spherical mirror and a perspective camera. Due to the symmetry of the sphere there exist symmetry of revolution wherever the perspective camera is located. The system is characterized by the radius of the sphere Rs, the distance from the camera to the center of the sphere Zs, the intrinsic parameters of the perspective camera and the relative rotation of the camera with respect to the axis of revolution. This axis is defined

Algebraic and metric distances

When evaluating if a point belongs to a line-image for fitting the curve is necessary a function measuring the distance from a point to the line-image. The quality of the extracted line-image depends on this distance. In this Section, we present a qualitative comparison among distances and we propose different approaches for tackling this problem. Having the line-image Eq. (9) a measure of distance is the algebraic distancedalg(x,L)=|Zr(l2xzl1yz)+(l¯1xz+l¯2yz)+l¯3rzcotϕ| which is measured in

Robust line extraction in non-central systems

In this Section, we present a method for line-image extraction in non-central systems. Line extraction in non-central systems is an unsolved challenging task due to the difficulties of mapping the 3D of the line on a 2D space, the sensitivity to noise in fitting, and the elevate number of degrees of freedom involved. Up to our knowledge, this is the first work addressing this problem.

Assuming the system is calibrated, four points of the line-image are needed to define a line-image and its

Experimental evaluation

In this section, we present the experimental evaluation of the proposal. First, we show simulations to study the behaviour of the proposed method. Then we present selected examples of line-extraction in synthetic and real images. Experiments with synthetic images allow testing the complete algorithm knowing the ground truth and with absence of calibration errors. The extraction method is also tested in images taken with real catadioptric systems. Finally we present several examples of

Conclusions

In this paper, we have analyzed the geometry of line projections in non-central catadioptric systems focusing on conical and spherical catadioptric cameras. In both cases line-images are quartic polynomials, although in the conical case it can be simplified to a compact description of 6 homogeneous parameters. The underlying structure of line-images has been used for designing a line-image extraction method and to obtain the geometry of the mirror in the case of the conical catadioptric camera.

Acknowledgments

This work was supported by the Spanish projects DPI2015-65962-R (MINECO/FEDER, UE) and DPI2014-61792-EXP.

References (38)

  • J. Barreto

    General Central Projection Systems: Modeling, Calibration and Visual Servoing.

    (2003)
  • J. Bermudez-Cameo et al.

    Minimal solution for computing pairs of lines in non-central cameras

    Proceedings of the Twelfth Asian Conference on Computer Vision (ACCV)

    (2014)
  • J. Bermudez-Cameo et al.

    Line-images in cone mirror catadioptric systems

    Proceedings of the Twenty-Second International Conference on Pattern Recognition (ICPR)

    (2014)
  • J. Bermudez-Cameo et al.

    Automatic line extraction in uncalibrated omnidirectional cameras with revolution symmetry

    Int. J. Comput. Vis.

    (2015)
  • V. Caglioti et al.

    On the localization of straight lines in 3D space from single 2D images

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    (2005)
  • V. Caglioti et al.

    Methods for space line localization from single catadioptric images: new proposals and comparisons

    Proceedings of the IEEE Eleventh International Conference on Computer Vision (ICCV)

    (2007)
  • V. Caglioti et al.

    Single-image calibration of off-axis catadioptric cameras using lines

    Proceedings of the International Conference on Computer Vision (ICCV)

    (2007)
  • ChumO. et al.

    Matching with prosac-progressive sample consensus

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    (2005)
  • S. Gasparini et al.

    Line localization from single catadioptric images

    Int. J. Comput. Vis.

    (2011)
  • Cited by (0)

    View full text