A new calibration method for line-structured light vision sensors based on concentric circle feature

Abstract

Background

Determining the relative ubiety between the camera and the laser projector in a line-structured light vision sensor is a classical yet important task. Typical calibration methods often confront problems, such as difficulty of producing the target precisely and introduction of perspective projection errors.

Methods

In this work, a new calibration method based on a concentric circle feature is introduced. The proposed method is based on geometrical properties and can reduce the perspective projection error. In our method, the vanishing line of the light plane is firstly deduced from the imaged concentric circles. Then the normal vector of the light plane is determined. Consequently, the complete expression can be confirmed from the principle of the intersecting planes.

Results and conclusion

The proposed method is simple and robustness as the basic theory is geometrical properties. Accuracy evaluation experiment shows that the accuracy of the calibration method can reach 0.07 mm within the view field of about 200 × 200 mm. This accuracy is comparable to the commonly used calibration method with a checkerboard planar target, whereas our target is simple to produce.

Background

The Line-Structured Light Vision Sensor (LSLVS) plays an important role in the field of industry measurement owing to its wide measurement range, high precision, real-time ability, simple information extracting, and so forth [1, 2]. The typical structure consists of one camera and one laser projector. Estimation of the relationship between the camera and the laser projector, which is called as calibration of LSLVS, is one of the most important tasks.

Heretofore, there are many calibration methods. According to ways of obtaining feature points on the structured light plane, these methods can be classified into three categories: three-dimensional (3-D) target method [3, 4], planar target (2-D) method [5,6,7], and one-dimensional (1-D) target method [8,9,10,11,12,13].

The 3D target method is based on the invariance of double cross-ratio. With a special calibration 3D target, enough non-collinear feature points on the structured light plane can be determined based on the theory of the invariance of double cross-ratio. Afterwards, relative ubiety between the camera and the structured light plane projector is confirmed. Different planar targets with different features are used according to different theories. In Ref. [10], a planar target that contains a pattern of squares is utilized to finish the calibration of LSLVS. The intersection points of the light stripe and calibration squares with exactly known size can be obtained under the image coordinate system (ICS). Accordingly, feature points on the structured light plane can be gained based on the invariance of cross-ratio. A 1-D target method is proposed owing to its convenient operation. The feature point, namely the intersection point of the light stripe and a 1D target, can be determined based on the invariance of cross-ratio. Therefore, enough feature points can be obtained from the random movements of the 1D target to different positions.

Unfortunately, the 3D target based method is not well suited for on-site calibration as there will be some inevitable problems, e.g., mutual occlusion between different planes of the target, the difficulty of producing the target precisely, the cumbersomeness of the target, etc. In the 1D target based method, few feature points are obtained and the calibration result is not precise enough. In comparison, the planar target-based method is with easy operation and satisfactory results but perspective projection error is inevitable.

In this paper, a planar target with a pattern of two (or more) concentric circles is utilized to finish the calibration of LSLVS. In our calibraiton algorithm, the vanishing line of the light plane is deduced from imaged circles, then normal vector of the light plane is confirmed. The complete expression can be confirmed from the intersecting planes under a camera coordinate system. In the application, we evaluate our algorithms and accurate results are achieved. The contribution of our paper is that we initiate a new method based on a concentric circles feature to calibrate LSLVS. The algorithm can reduce the perspective deviation with a satisfying result. Moreover, the planar target used in our method is easy to make precisely, and meanwhile, the proposed method is efficient and convenient, especially for onsite calibration.

Measurement model

The location relationship between the camera in LSLVS and the structured-light plane projector remains unchangeable in the process of calibration and measurement. So the structured-light plane can be expressed as a fixed function, which is defined as Eq.(1) under camera coordinate system.

Consider.

$$ \left[{A}_L,{B}_L,{C}_L,{D}_L\right]{\left[X,Y,Z,1\right]}^{\mathrm{T}}=0 $$

(1)

where A_L, B_L, C_L, D_L are parameters of the structured-light plane’s expression. The measurement model of LSLVS is illustrated in Fig. 1. O-XYZ is the Camera Coordinate System (CCS) while o-xy is the Image Coordinate System (ICS). Under CCS, the center of projection of the camera is at the origin and the optical axis points in the positive Z direction. A spatial point P is projected onto the plane with Z = f₀, referred to as the image plane under the CCS, where f₀ is the effective focal length (EFL). Supposing p = (x, y, 1)^T is the projection of P = (X, Y, Z)^T on the image plane.

Fig. 1

Measurement model of LSLVS

Under the idealized pinhole imaging model, the ideal model of the camera, P, p and the center of projection O are collinear. The fact can be expressed by the following equation:

$$ Z\left[\begin{array}{c}x\\ {}y\\ {}1\end{array}\right]=\left[\begin{array}{cccc}{f}_0& 0& 0& 0\\ {}0& {f}_0& 0& 0\\ {}0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}X\\ {}Y\\ {}Z\\ {}1\end{array}\right] $$

(2)

Practically, the radial distortion and the tangential distortion of the lens are inevitable. When considering the radial distortion, we have the following equations:

$$ \left\{\begin{array}{c}\overline{x}=x\left(1+{k}_1{r}^2+{k}_2{r}^4\right)\\ {}\overline{y}=y\left(1+{k}_1{r}^2+{k}_2{r}^4\right)\end{array}\right. $$

(3)

where r² = x² + y², (x, y)^T is the distorted image coordinate, and ${\left(\overline{x,}\overline{y}\right)}^T$ is the idealized one, k₁, k₂ are the radial distortion coefficients of the lens.

Properties of concentric circles

For the purpose of easy description, we define a world coordinate system, the XY plane of which is the target plane. In this case, the z-axis of the world coordinate system (WCS) is perpendicular to the target plane (as illustrated in Fig. 2). Also, we define the O-XY of WCS on the target plane as the target plane coordinate system (TCS). Define C₁ and C₂ as two concentric circles, and their circle center is O. Their corresponding images are C₁, C₂ and o.

Fig. 2

(a) Relative position of Concentric circles and (b) its relative coordinate system

Without loss of generality, define the homogenous representation of the circles center as [x₀, y₀, 1]^T under TCS and the radii as r₁ and r₂. So the matrix representation of C₁ is.

$$ {C}_1=\left[\begin{array}{ccc}1& 0& -{x}_0\\ {}0& 1& -{y}_0\\ {}-{x}_0& -{y}_0& {x}_0^2+{y}_0^2-{r}_1^2\end{array}\right] $$

(4)

while the representation of C₂ is.

$$ {C}_2=\left[\begin{array}{ccc}1& 0& -{x}_0\\ {}0& 1& -{y}_0\\ {}-{x}_0& -{y}_0& {x}_0^2+{y}_0^2-{r}_2^2\end{array}\right] $$

(5)

The TCS is not unique when the concentric circles are described as Eq.(4) and Eq.(5). Under any one coordinate system whose origin coincides with the center of these concentric circles, expressions of these concentric circles are the same.

Define the homography matrix from the image plane to the target plane where concentric circles locates in as H. The matrix experiment of a circle can be expressed as.

$$ {p}^T Cp=0. $$

(6)

Where p is the homography expression of the point located on the circle.

So we have.

$$ \left\{\begin{array}{c}{p_1}^T{C}_1{p}_1=0\\ {}p{{\hbox{'}}_1}^T{C}_1^{\hbox{'}}p{\hbox{'}}_1=0\end{array}\right.. $$

(7)

As ${p}_1^{\hbox{'}}={Hp}_1$, Eq.(7) can be rewritten as.

$$ \left\{\begin{array}{c}{p_1}^T{C}_1{p}_1=0\\ {}{p_1}^T{H}^T{C}_1^{\hbox{'}}{Hp}_1=0\end{array}\right. $$

(8)

The the realtion between C₁ and ${C}_1^{\hbox{'}}$ is obtained:

$$ {C}_1={H}^T{C}_1^{\hbox{'}}H $$

(9)

So we have.

$$ \left\{\begin{array}{c}{C}_1^{\hbox{'}}={H}^{-T}{C}_1{H}^{-1}\\ {}{C}_2^{\hbox{'}}={H}^{-T}{C}_2{H}^{-1}\end{array}\right. $$

(10)

Then.

$$ {C_2^{\hbox{'}}}^{-1}{C}_1^{\hbox{'}}=H\left({C}_2^{-1}{C}_1\right){H}^{\hbox{-} 1} $$

(11)

The eigen-decomposition can be expressed as.

$$ \left[V,D\right]= eig\left({C}_2,{C}_1\right) $$

(12)

where $V=\left[\begin{array}{ccc}1& 0& {x}_0\\ {}0& 1& {y}_0\\ {}0& 0& 1\end{array}\right]$ and $D=\left[\begin{array}{ccc}1& 0& 0\\ {}0& 1& 0\\ {}0& 0& {r}_1^2/{r}_2^2\end{array}\right]$. From matrix D, we find that ${C}_2^{\hbox{'}-1}{C}_1^{\hbox{'}}$ has three eigenvalues, of which two are identical and one is different. From matrix V, we find that the corresponding eigenvectors of the identical eigenvalues are [1 0 0]^T, [0 1 0]^T, which are points on the infinity line. We also find the corresponding eigenvector of the different eigenvalue is [x₀ y₀ 1]^T, which is the circle center. Based on the above analysis, we can conclude that the circle center and the line at infinity can be recovered by the eigenvectors of the matrix ${C}_2^{-1}{C}_1$ [14].

Eq.(11) is rewritten as.

$$ {C}_2^{\hbox{'}-1}{C}_1^{\hbox{'}}H={HC}_2^{-1}{C}_1 $$

(13)

As the world coordinate system can be defined freely, the concentric circles can be expressed as Eq.(4) and Eq.(5). Eq.(13) is the form of AX = XB, which is widely used in robot hand-eye calibration [15]. The general solutions of Eq.(8) can be calculated easily. One solution is corresponding to one coordinate system (TCS). So any one of the solutions can be chosen, then the matrix H transformed from the image plane to the target plane (under TCS) can be confirmed.

Calibration

In our method, a planar target with two concentric circles is designed to finish the calibration. As is known, the projection of a circle on a plane is an ellipse (as illustrated in Fig. 3), which can be expressed as [16, 17].

$$ {\left[\begin{array}{c}x\\ {}y\\ {}1\end{array}\right]}^{\mathrm{T}}\left[\begin{array}{ccc}a& c/2& d/2\\ {}c/2& b& e/2\\ {}d/2& e/2& f\end{array}\right]\left[\begin{array}{c}x\\ {}y\\ {}1\end{array}\right]=0 $$

(14)

Fig. 3

Projection of circle on image plane

As described in Eq.(11), matrix ${C}_2^{-1}{C}_1$ is similar with the matrix ${C}_2^{\hbox{'}-1}{C}_1^{\hbox{'}}$, defining the eigenpair of ${C}_2^{-1}{C}_1$ as (λ, x). According to the property of similarity transformation, the eigenpair of ${C}_2^{\hbox{'}-1}{C}_1^{\hbox{'}}$ is (λ, Hx). As is known, the vanishing point is the image of the infinity point and the vanishing point must be located on the vanishing line, which is the image of the infinity line. When projections of the two concentric circles (${C}_1^{\hbox{'}}$ and ${C}_2^{\hbox{'}}$) are determined, the image circle center and the vanishing line of the target plane can be deduced from their eigenvectors.

Clearly, the light stripe locates on the target plane. As one plane has only one vanishing line, the intersection point of the vanishing line of the target plane and the light stripe is a vanishing point of the light plane. So when we place the target in different positions, more vanishing points will be obtained. Then the vanishing line of the light plane is confirmed (as illustrated in Fig. 4). In Fig. 4, L_T is the vanishing line of the target plane. P_n is the vanishing point of the target plane and the light plane. L_l is the vanishing line of the light plane.

Fig. 4

The interrelationship between vanishing line and vanishing points in different positions

The vanishing line and its corresponding plane can be related by the following equation.

$$ l={K}^{-T}\overrightarrow{n} $$

(15)

where K is the intrinsic parameter matrix of the camera, $\overrightarrow{n}$ is the normal vector of the corresponding plane and l is the homogeneous coordinate of the vanishing line. Based on Eq.(15), the normal vector of the light plane can be deduced from its vanishing line [18].

$$ \overrightarrow{n}={K}^Tl $$

(16)

when we obtain the homography matrix from image plane to the target plane, the rotation matrix and the translation matrix from the camera coordinate system to the world coordinate system can be decomposed from the homography matrix. The light stripe is similarly located on the light plane, the target plane and its back projection plane under CCS. When we define the target plane as.

$$ \left[{A}_P,{B}_P,{C}_P,{D}_P\right]{\left[X,Y,Z,1\right]}^{\mathrm{T}}=0 $$

(17)

And the back projection plane is.

$$ \left[{A}_J,{B}_J,{C}_J,{D}_J\right]{\left[X,Y,Z,1\right]}^{\mathrm{T}}=0 $$

(18)

Then the series of planes crossing the light stripe on the target can be expressed as.

$$ \left[{A}_P+\mathrm{k}{A}_J,{B}_P+\mathrm{k}{B}_J,{C}_P+\mathrm{k}{C}_J,{D}_P+\mathrm{k}{D}_J\right]{\left[X,Y,Z,1\right]}^{\mathrm{T}}=0 $$

(19)

Eq.(19), k is a scale factor. As the normal vector of the light plane crossing the plane stripe is known, the parameter D_L can be easily be confrimed from Eq.(19) and Eq.(1).

Briefly, the calibration procedure is given as follow:

Step 1: Extract concentric circles images ${C}_1^{\hbox{'}}$ and ${C}_2^{\hbox{'}}$.

Step 2: Calculate the eigenvectors of ${C}_2^{\hbox{'}-1}{C}_2^{\hbox{'}}$, then the vanishing line and the imaged circle center can be confirmed.

Step 3: Confirm the normal vector of the light plane from its vanishing line.

Step 4: Confirm the homography matrix from the target plane to the image plane. Then the rotation matrix and translation matrix from the world coordinate system to the camera coordinate system is deduced.

Step 5: The back projection plane can be confirmed based on the light strip on the target plane.

Step 6: Parameter D_L which is defined in Eq.(1) can be confirmed from Eq.(19).

Simulation

In our computer simulations, we assumed the simulated camera has been calibrated, i.e. the intrinsic parameters are known: the focal lengthf = 1200, the scale factor in the x-coordinate direction (f_x) is equal to the scale factor in the y-coordinate direction (f_y), i.e. f_x = f_y = 1200, the principal point(u0, v0) = (400, 300). The expression of the light plane under the camera coordinate system is.

$$ \left[1.2,2.2,3.6,4.9\right]{\left[X,Y,Z,1\right]}^{\mathrm{T}}=0 $$

(20)

The pattern on the plane target contains two concentric circles with a radius of 25 mm and 20 mm respectively.

Influence of image noise on calibration accuracy

In our simulations, the target is moved to 5 different positions. 200 feature points on each circle image are chosen to fit the ellipse and the least squares ellipse fitting algorithm is utilized. The light strip is fitted by 30 image points. Gaussian noise with standard deviations varying from 0 to 1.0 pixels is added to both coordinates of the image points to generate the perturbed image points.

In the proposed algorithm, normal vector of the light plane is obtained from its vanishing line and the parameter D_L is deduced from the back projection plane. Errors of the normal vector and the parameter D of the structured light plane are illustrated in Fig. 5. The root mean square error (RMS) and the mean absoulte error (MAE) are illustrated. Errors illustrated in Fig. 5(a) are soloved from the intersection angle of two nornal vectors, the idealized light plane’s and the perturbed one’s. Similarly, errors illustrated in Fig. 5(b) are difference from idealized parameter D_L and the calculated one.

Fig. 5

(a) Error of the normal vector as a function of the standard deviation of additive Gaussian noise, (b) Error of parameter D as a function of the standard deviation of additive Gaussian noise

Each point in Fig. 5 represents result averaged 200 uniformly distributed rotations. From Fig. 5, we can see that errors increase over the noise level, including root mean square error and mean absoulte error.

Influence of circle number on calibration accuracy

We generalize our results to three concentric circles and four concentric circles. In this experiment, we choose 200 points to fit the ellipse and also place the target to 5 different positions. Calibration results are illustrated in Fig. 6 and Fig. 7. The related notations are the same as mentioned in Fig. 5.

Fig. 6

(a) Error of normal vector as a function of the standard deviation of the Gaussian noise when three concentric circles is used, (b) Error of parameter D as a function of the standard deviation of the Gaussian noise when three concentric circles is used

Fig. 7

(a)Error of normal vector as a function of the standard deviation of the Gaussian noise when four concentric circles is used, (b) Error of parameter D as a function of the standard deviation of the Gaussian noise when four concentric circles is used

As illustrated in Fig. 6 and Fig. 7, the calibration result is better as the number of concentric circles increases. When more concentric circles are used, the vanishing line and the homography matrix have a precise result. In this case, calibration result is more accuracy.

Results and discussion

Camera calibration

In the real scene experiment, the camera used to capture the image is AVT F-504B with a resolution of 2452 × 2056 pixels and the view field is about 200 × 200 mm (as illustrated in Fig. 8). The laser projector is a line laser. A planar target with two concentric circles is made and the radii of the two circles are 50 mm and 70 mm respectively. Its machining accuracy is 0.01 mm.

Fig. 8

Structure of experiment system

The camera can be calibrated by Zhang’s calibration method [19, 20] or Huang’s calibration method [16]. Based on Huang’s calibration method, intrinsic parameters can be obtained simultaneously when we finish the calibration of LSLVS using our target. In our experiment, we calibrate the camera by Zhang’s method. The checkerboard-pattern target is produced with accuracy of 10 μm. Captured images are illustrated in Fig. 9 and the intrinsic parameters are listed in Table 1.

Fig. 9

Images used for the calibration of intrinsic parameters in different positions (a-f)

Table 1 Intrinsic parameters of the camera

In Table 1, f_x is the scale factor in the x-coordinate direction, f_y is the scale factor in the y-coordinate direction, (u₀, v₀)^T is the coordinates of the principal point. k₁ and k₂ are the distortion coefficients of lens.

Sensor calibration

A planar target with two concentric circles is used to calibrate LSLVS, and the radii of the two circles are 50 mm and 70 mm respectively. Its produced accuracy is 0.01 mm. The target is placed to 5 different positions. We first compensated for camera distortion by rectifying all real images, and then the image of concentric circles is extracted [21] in sub-pixel precision, which is illustrated in Fig. 10.

Fig. 10

Extraction of imaged concentric circles in four different positions (a-d)

LSLVS is calibrated by the method mentioned above respectively. Function of the structured light plane under camera coordinate system is expressed as.

$$ x+0.013y+0.390z-518.151=0 $$

(21)

Moreover, we used a target with three concentric circles to finish the calibration task. The extraction of imaged concentric circles are illustrated in Fig. 11.

Fig. 11

Extraction of imaged concentric circles in four different positions (a-d)

Function of the structured light plane under camera coordinate system is expressed as.

$$ x+0.012y+0.381z-515.237=0 $$

(22)

Accuracy evaluation

A planar checkerboard-pattern target is used to evaluate the accuracy of the proposed calibration method. As the side length of each checkerboard is known exactly, the coordinate of the feature points under TCS can be solved based on the invariance of cross-ratio. The theory is descirbed as follows:

The grid pitch of the target is known accurately as l while the length of AD can be defined as l₀ (see Fig. 12). Based on the invariance of cross-ratio, the following equation can be obtained:

$$ \frac{AB/ BD}{AC/ DC}=\frac{2l/\left(2l-{l}_0\right)}{3l/\left(3l-{l}_0\right)} $$

(23)

Fig. 12

The determination of real length

The real length of AD can be solved, so can A₁D₁. Then the distance between point D and point D₁ can be worked out, i.e. the distance between each of the adjacent feature points (dTru) can be obtained, which can be treated as the true value owing to its high accuracy.

All measurement values (dMea) are compared with their corresponding true value. As the target can be moved into different positions randomly, we can obtain enough distances to evaluate our calibration result, ten of which are listed in Table 2.

Table 2 Evaluation of our calibration results

In Table 2, dTru denotes the real distance between two feature points, while dMea is the measurement result based on the calibration result using the proposed method. As listed in Table 2, the root mean square error of the calibration result (RMS) obtained by our proposed method is 0.072 mm. The calibration results are precise enough as two concentric circles are used and the target is just placed to four different positions.

Comparisons & Discussions

There are many calibration methods for line structure light vision sensor. Three methods with different targets are concisely described as follows:

• A. The method based on the invariance of double cross-ratio (named 3D method) [7]. The utilized 3D target consists of two rigid planes that are perpendicular to each other, and squares and/or rectangles exist on each plane. Enough features points can be obtained based on the invariance of double cross-ratio. The root mean square error of the measurement is less than 0.151 mm as mentioned in Ref. [7] within the field of about 300×300 mm.

• B. The planar target based method (named 2D method) [10] approach uses a planar target with a pattern of 3×3 squares to calibrate the LSLVS, the size of each square is 35×35 mm. The intersection points of the light stripe and calibration squares with exactly known size can be obtained under the image coordinate system. Accordingly, feature points on the structured light plane can be gained based on the invariance of cross-ratio. The root mean square error of 30 distances as mentioned in the experiment of [10] is 0.085 mm within the field of about 200×200 mm.

• C. The 1D target based method (named 1D method) [8] in the calibration approach, has a target length of about 400 mm. And six small holes, the distance of each adjacent pair is 40 mm, are located on the target. One feature point can be determined based on the invariance of cross-ratio each time. Enough feature points can be obtained to calibrate the LSLVS from the random movements of the 1D target. The root mean square error of the measurement can reach 0.065 mm in [8] within the field of about 300×200 mm. These corresponding target are illustrated in Fig. 13.

  Fig. 13

  

  Targets of these described calibration methods in their paper. (a) is the 3D target in Ref. [7], (b) is the planar target in Ref. [10], while (c) is the 1D target in Ref. [8]

As metioned above, the 3D target based method is not well suited for on-site calibration as some inevitable problems, e.g., mutual occlusion between different planes of the target, the difficulty of producing precisely, the cumbersomeness of the target, etc. Comparatively, the 2D method and 1D method is more suitable for on-site calibration. But as the feature points of the 1D target is less than the 2D target, the target should be moved to more positions to finish the calibration (or get an accurate result). Therefore, the 2D method is more convenient and is popularly utilized. In this case, different planar targets appeared.

As the most typical calibration method for line sturctured light vision sensor is the planar target based method [10], we employ the method in Ref. [10] to calibrate the sensor. The target is with a circle array pattern and its machining accuracy is 0.01 mm. Images used to calibrate the line structure light sensor are displayed in Fig. 14. The calibration result is:

$$ x+0.012y+0.389z-516.312=0 $$

(24)

Fig. 14

Images used to calibrate the sensor in Zhou’s method. (a)-(f) are images of the target in different positions captured by the camera

Based on the evaluation method described above, the results of Zhou’s calibration method are listed in Table 3.

Table 3 Evaluation of Zhou’s calibration results

The 3D coordinate of these feature points obtained according these calibration methods are illustrated in Fig. 15.

Fig. 15

Feature points obtained according to different calibration methods

In Fig. 15, the method with two circles is the proposed calibration method using a planar target with two concentric circles, while the method with three circles is the proposed calibration method using a planar target with three concentric circles. Compared with the calibration results listed in Table 2, we can get the conclusion that the accuracy of our proposed calibration method is comparable to the method which uses a planar target.

As described above, the precision of these described calibration methods are listed in Table 4.

Table 4 Comparison of calibration results

Our proposed calibration method has nearly the same accuracy with the 2D method and the 1D method, and better than the 3D method. Furthermore, as the target utilized in our proposed approach is a plane with several circles, it is easier to produce precisely than the target used in the 2D method. Although the 1D method can accomplish the on-site calibration of LSLVS conveniently, the target has to move to more positions than our proposed approach in order to obtain enough feature points. Yet the feature points of the 1D target only have the constraint of length, which may affect the accuracy of calibration. Neither production difficulty nor length constraint exists in our proposed approach, so it is more suitable for on-site calibration of LSLVS compared with the three typical calibration methods. Furthermore, the related feature information is summarized as listed in Table 5.

Table 5 Comparison of calibration features

Conclusions

In this paper, a calibration method based on properties of concentric circles is described. A planar target with a pattern of several concentric circles is utilized to finish the calibration of LSLVS. The normal vector of the light plane is deduced from its vanishing line. Then the parameter D is confirmed by the back projection plane deduced based on the light strip on the target plane.

The contribution of our paper is that we initiate a new method based on concentric circles to calibrate LSLVS. The introduced method can reduce the perspective deviation and obtain a precise result. Moreover, the planar target used in our calibration method is easy to make precisely, meanwhile, the proposed method is efficient and convenient, especially for onsite calibration.