The projective reconstruction of points, lines, quadrics, plane conics and degenerate quadrics using uncalibrated cameras
Introduction
We begin by citing relevant work on multiple view uncalibrated reconstruction. Hartley [1] and Hartley and Zisserman [2] have proposed the use of n-view tensors which can be computed directly from image features. Carlsson and Weinshall [3] have used an approach based on the duality principle between points and camera centers. Rother et al. [4] further simplified this scheme, by relying on the a priori knowledge of planarity in the scene. Planarity has also been used by Kaucic et al. [5] to simplify the computation of n-view tensors. Projective factorization alone has also been used by Sturm and Triggs in [6]; however, this approach is only valid if the projective depths axe known or can be reasonably guessed. Another approach proposed by Bayro and Banarer [7] produces a projective reconstruction using the n-view tensors and the invariants theory. In [8] Kahl and Heyden propose a method to compute structure and motion from points, lines and conics; but their approach is based on affine cameras and factorization—though a method for handling missing data is introduced.
Bundle Adjustment (BA) [9] can be used to simultaneously estimate multiple cameras and 3D primitives. Classical Bundle Adjustment [2] has been improved by McLauchlan [10] to account for gauge invariance. Bartoli and Sturm [11] developed a minimal parameterization scheme and Malis and Bartoli [12] further improved this to produce an Euclidean reconstruction. All this work is based in the estimation of cameras and 3D points. Berthilsson et al. [13] used BA for the estimation of general space curves. Shan et al. [14] applied BA to model human faces from a prototype neutral face model and a series of images.
In [15] Åström proposed a method based on Bundle Adjustment for the computation of structure and motion from points, lines and conics. The formulation is somewhat different to the one presented in this paper (lines are represented by two endpoints, the derivatives are different too) and the problem of epipolar consistency in quadrics is not properly addressed. Cross [16] has already shown that the epipolar consistency check is a problem that requires attention since it is critical for the reconstruction of topologically-consistent quadrics and conics. We have properly addressed that problem in this paper and also present an improvement over Cross’ work. In addition, in [15] only conics are considered, whereas we also consider quadrics (both full-rank and degenerate).
On the reconstruction of quadrics, Cross and Zisserman [17] work with previously known camera matrices and develop an algorithm to do outline correction of the conics and perform the reconstruction from two views. No clear details are given on how to extend the algorithm for three or more views, and nothing is said about the problem of noisy camera matrices. In [16], Cross describes a similar method for outline correction and reconstruction of quadrics from n views. However, the details are only given for two and three views; the algorithm for four or more views cannot be directly applied as stated by Cross. Again, camera matrices are assumed to be known beforehand without error. As stated earlier, we will present an improvement over Cross’ outline correction algorithm for n views.
As we can see, most of these approaches concentrate on a specific type of primitive. To the best of our knowledge, there is not a solution, for the case of projective cameras, that takes into account all the types of primitives at the same time while keeping the topology of quadrics. In this paper we propose a method that simultaneously computes 3D points, lines, quadrics, plane conics, degenerate quadrics and cameras, from a series of uncalibrated views without resorting to projective factorization (i.e. no need to know the projective depths) or n-view tensor estimation. Furthermore, no constraints (like planarity) are imposed on the scene and the features are not required to be visible in all views.
We must also stress that the reconstruction of quadrics can be quite tricky in the presence of noise. In this case, the topology of the quadrics can be destroyed by the noise if no measures are taken to correct this situation. We have therefore considered the need to correct the topology of the quadrics and also present an original algorithm for the optimal correction of outlines in four or more views. It must be noted that an earlier version of the work presented in this paper was accepted in [18], but in that paper, only full-rank quadrics are considered and the novel algorithm for outline correction just mentioned is not presented there.
Section snippets
Projection of geometric entities
A review on the projection of points, lines and quadrics will be given in this section. The interested reader may look in [2] for further details.
Initial projective reconstruction
Bundle Adjustment will be used for the simultaneous computation of cameras and structure. Briefly, this method consists in using an iterative minimization algorithm, such as the Newton or the Levenberg–Marquardt methods, to refine an initial solution. The minimization is done according to a cost function which usually measures the geometric distance between the projected points and the measured data. In other words, this method requires an initial reconstruction of the scene and the cameras. At
Refinement of the reconstruction
In the previous section we described how we make our initial reconstruction. This reconstruction is used as a seed for a Bundle Adjustment algorithm that refines the solution. Briefly, the classical algorithm for the reconstruction of points and cameras consists in minimizingwhere is set of estimated cameras, (is the set of estimated 3D points, xi,j is the set of measured 2D features and d(x,y) is the geometric distance between points x and y. Usually, this
Experimental analysis
We conducted four experiments where some objects displaying points, lines, quadrics, plane conics and degenerate quadrics were reconstructed. In the first experiment, a few objects were placed in a box and some images were taken with the stereo rig of our robot, but only the left image was used (see Fig. 10(a–c)). In the second and third experiments, various objects were also placed in boxes but we used a commercial digital camera (see Fig. 11, Fig. 12(a–c)). For the last experiment, a fountain
Conclusions
We have presented a method that simultaneously estimates points, lines, quadrics, plane conics, degenerate quadrics and cameras. In contrast, previous work on simultaneous reconstruction of multiple types of primitives has not considered quadrics at all. Additionally, our method takes into account the epipolar consistency check during bundle adjustment. To the best of our knowledge, this check has not been introduced inside the bundle adjustment iterations. Another advantage of our method is
References (20)
Computation of the quadrifocal tensor
(1998)- et al.
Multiple View Geometry in Computer Vision
(2000) - et al.
Dual computation of shape and camera positions from multiple images
International Journal of Computer Vision
(1998) - et al.
Linear multi view reconstruction and camera recovery using a reference plane
International Journal of Computer Vision (Special Issue on Multi-View Modeling and Rendering of Visual Scenes)
(2002) - et al.
Plane-based projective reconstruction
(2001) - et al.
A factorization based algorithm for multi-image projective structure and motion
(1996) - et al.
A geometric approach for the theory and applications of 3d projective invariants
Journal of Mathematical Imaging and Vision
(2002) - et al.
Affine structure and motion from points, lines and conics
International Journal of Computer Vision
(1999) - et al.
Bundle adjustment—a modern synthesis
(1999) Gauge invariance in projective 3d reconstruction
(1999)
Cited by (7)
Factorization based structure from motion with object priors
2018, Computer Vision and Image UnderstandingCitation Excerpt :As our first contribution, the proposed semantic factorization framework for SfM provides a description of objects in terms of 3D conics, specifically quadrics, which can be embedded in a compact bilinear model. As a peculiar aspect, up to now, most SfM methods mostly deals with points, while very few exceptions exist in the literature using different geometrical entities such as lines (Bartoli and Sturm, 2005; Matinec and Pajdla, 2003; Quan and Kanade, 1996; Schindler et al., 2006), generic curves (Berthilsson et al., 2001; Kaess et al., 2004; Kahl and August, 2003; Mai and Hung, 2010; Nurutdinova and Fitzgibbon, 2015) and conics (Cross and Zisserman, 1998; Kahl and Heyden, 1999; Reyes and Bayro Corrochano, 2005). Lines were the first geometric entities included in standard 3D reconstruction approaches.
A Geometric Model for Specularity Prediction on Planar Surfaces with Multiple Light Sources
2018, IEEE Transactions on Visualization and Computer GraphicsProbabilistic Structure from Motion with Objects (PSfMO)
2017, Proceedings of the IEEE International Conference on Computer VisionAn Empirical Model for Specularity Prediction with Application to Dynamic Retexturing
2016, Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016Structure from Motion with Objects
2016, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern RecognitionMultiview Differential Geometry of Curves
2016, International Journal of Computer Vision