Elsevier

Image and Vision Computing

Volume 23, Issue 8, 1 August 2005, Pages 693-706
Image and Vision Computing

The projective reconstruction of points, lines, quadrics, plane conics and degenerate quadrics using uncalibrated cameras

https://doi.org/10.1016/j.imavis.2005.03.008Get rights and content

Abstract

In this paper we present an algorithm for the simultaneous projective reconstruction of points, lines, quadrics, plane conics and degenerate quadrics using Bundle Adjustment. In contrast, most existing work on projective reconstruction focuses mainly on one type of primitive. Furthermore, for the reconstruction of quadrics (both full-rank and degenerate) and plane conics, a novel algorithm for the rectification of the outlines projected in n views is presented. Finally, we found out that due to the noise in the data, the global minimum of the classic cost function used in Bundle Adjustment may unfortunately break the topology of the quadrics. Hence, during the iterations of this method, we need to add the constraints to keep the quadrics within acceptable limits to preserve their topology.

Introduction

We begin by citing relevant work on multiple view uncalibrated reconstruction. Hartley [1] and Hartley and Zisserman [2] have proposed the use of n-view tensors which can be computed directly from image features. Carlsson and Weinshall [3] have used an approach based on the duality principle between points and camera centers. Rother et al. [4] further simplified this scheme, by relying on the a priori knowledge of planarity in the scene. Planarity has also been used by Kaucic et al. [5] to simplify the computation of n-view tensors. Projective factorization alone has also been used by Sturm and Triggs in [6]; however, this approach is only valid if the projective depths axe known or can be reasonably guessed. Another approach proposed by Bayro and Banarer [7] produces a projective reconstruction using the n-view tensors and the invariants theory. In [8] Kahl and Heyden propose a method to compute structure and motion from points, lines and conics; but their approach is based on affine cameras and factorization—though a method for handling missing data is introduced.

Bundle Adjustment (BA) [9] can be used to simultaneously estimate multiple cameras and 3D primitives. Classical Bundle Adjustment [2] has been improved by McLauchlan [10] to account for gauge invariance. Bartoli and Sturm [11] developed a minimal parameterization scheme and Malis and Bartoli [12] further improved this to produce an Euclidean reconstruction. All this work is based in the estimation of cameras and 3D points. Berthilsson et al. [13] used BA for the estimation of general space curves. Shan et al. [14] applied BA to model human faces from a prototype neutral face model and a series of images.

In [15] Åström proposed a method based on Bundle Adjustment for the computation of structure and motion from points, lines and conics. The formulation is somewhat different to the one presented in this paper (lines are represented by two endpoints, the derivatives are different too) and the problem of epipolar consistency in quadrics is not properly addressed. Cross [16] has already shown that the epipolar consistency check is a problem that requires attention since it is critical for the reconstruction of topologically-consistent quadrics and conics. We have properly addressed that problem in this paper and also present an improvement over Cross’ work. In addition, in [15] only conics are considered, whereas we also consider quadrics (both full-rank and degenerate).

On the reconstruction of quadrics, Cross and Zisserman [17] work with previously known camera matrices and develop an algorithm to do outline correction of the conics and perform the reconstruction from two views. No clear details are given on how to extend the algorithm for three or more views, and nothing is said about the problem of noisy camera matrices. In [16], Cross describes a similar method for outline correction and reconstruction of quadrics from n views. However, the details are only given for two and three views; the algorithm for four or more views cannot be directly applied as stated by Cross. Again, camera matrices are assumed to be known beforehand without error. As stated earlier, we will present an improvement over Cross’ outline correction algorithm for n views.

As we can see, most of these approaches concentrate on a specific type of primitive. To the best of our knowledge, there is not a solution, for the case of projective cameras, that takes into account all the types of primitives at the same time while keeping the topology of quadrics. In this paper we propose a method that simultaneously computes 3D points, lines, quadrics, plane conics, degenerate quadrics and cameras, from a series of uncalibrated views without resorting to projective factorization (i.e. no need to know the projective depths) or n-view tensor estimation. Furthermore, no constraints (like planarity) are imposed on the scene and the features are not required to be visible in all views.

We must also stress that the reconstruction of quadrics can be quite tricky in the presence of noise. In this case, the topology of the quadrics can be destroyed by the noise if no measures are taken to correct this situation. We have therefore considered the need to correct the topology of the quadrics and also present an original algorithm for the optimal correction of outlines in four or more views. It must be noted that an earlier version of the work presented in this paper was accepted in [18], but in that paper, only full-rank quadrics are considered and the novel algorithm for outline correction just mentioned is not presented there.

Section snippets

Projection of geometric entities

A review on the projection of points, lines and quadrics will be given in this section. The interested reader may look in [2] for further details.

Initial projective reconstruction

Bundle Adjustment will be used for the simultaneous computation of cameras and structure. Briefly, this method consists in using an iterative minimization algorithm, such as the Newton or the Levenberg–Marquardt methods, to refine an initial solution. The minimization is done according to a cost function which usually measures the geometric distance between the projected points and the measured data. In other words, this method requires an initial reconstruction of the scene and the cameras. At

Refinement of the reconstruction

In the previous section we described how we make our initial reconstruction. This reconstruction is used as a seed for a Bundle Adjustment algorithm that refines the solution. Briefly, the classical algorithm for the reconstruction of points and cameras consists in minimizingminPˆi,Xˆji,jd(PˆiXˆj,xi,j)2,where Pˆi is set of estimated cameras, Xˆj (is the set of estimated 3D points, xi,j is the set of measured 2D features and d(x,y) is the geometric distance between points x and y. Usually, this

Experimental analysis

We conducted four experiments where some objects displaying points, lines, quadrics, plane conics and degenerate quadrics were reconstructed. In the first experiment, a few objects were placed in a box and some images were taken with the stereo rig of our robot, but only the left image was used (see Fig. 10(a–c)). In the second and third experiments, various objects were also placed in boxes but we used a commercial digital camera (see Fig. 11, Fig. 12(a–c)). For the last experiment, a fountain

Conclusions

We have presented a method that simultaneously estimates points, lines, quadrics, plane conics, degenerate quadrics and cameras. In contrast, previous work on simultaneous reconstruction of multiple types of primitives has not considered quadrics at all. Additionally, our method takes into account the epipolar consistency check during bundle adjustment. To the best of our knowledge, this check has not been introduced inside the bundle adjustment iterations. Another advantage of our method is

References (20)

  • R. Hartley

    Computation of the quadrifocal tensor

    (1998)
  • R. Hartley et al.

    Multiple View Geometry in Computer Vision

    (2000)
  • S. Carlsson et al.

    Dual computation of shape and camera positions from multiple images

    International Journal of Computer Vision

    (1998)
  • C. Rother et al.

    Linear multi view reconstruction and camera recovery using a reference plane

    International Journal of Computer Vision (Special Issue on Multi-View Modeling and Rendering of Visual Scenes)

    (2002)
  • R. Kaucic et al.

    Plane-based projective reconstruction

    (2001)
  • P. Sturm et al.

    A factorization based algorithm for multi-image projective structure and motion

    (1996)
  • E. Bayro et al.

    A geometric approach for the theory and applications of 3d projective invariants

    Journal of Mathematical Imaging and Vision

    (2002)
  • F. Kahl et al.

    Affine structure and motion from points, lines and conics

    International Journal of Computer Vision

    (1999)
  • B. Triggs et al.

    Bundle adjustment—a modern synthesis

    (1999)
  • P.F. McLauchlan

    Gauge invariance in projective 3d reconstruction

    (1999)
There are more references available in the full text version of this article.

Cited by (7)

  • Factorization based structure from motion with object priors

    2018, Computer Vision and Image Understanding
    Citation Excerpt :

    As our first contribution, the proposed semantic factorization framework for SfM provides a description of objects in terms of 3D conics, specifically quadrics, which can be embedded in a compact bilinear model. As a peculiar aspect, up to now, most SfM methods mostly deals with points, while very few exceptions exist in the literature using different geometrical entities such as lines (Bartoli and Sturm, 2005; Matinec and Pajdla, 2003; Quan and Kanade, 1996; Schindler et al., 2006), generic curves (Berthilsson et al., 2001; Kaess et al., 2004; Kahl and August, 2003; Mai and Hung, 2010; Nurutdinova and Fitzgibbon, 2015) and conics (Cross and Zisserman, 1998; Kahl and Heyden, 1999; Reyes and Bayro Corrochano, 2005). Lines were the first geometric entities included in standard 3D reconstruction approaches.

  • A Geometric Model for Specularity Prediction on Planar Surfaces with Multiple Light Sources

    2018, IEEE Transactions on Visualization and Computer Graphics
  • Probabilistic Structure from Motion with Objects (PSfMO)

    2017, Proceedings of the IEEE International Conference on Computer Vision
  • An Empirical Model for Specularity Prediction with Application to Dynamic Retexturing

    2016, Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016
  • Structure from Motion with Objects

    2016, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • Multiview Differential Geometry of Curves

    2016, International Journal of Computer Vision
View all citing articles on Scopus
View full text