Implementation of an Optical Measurement Method for Monitoring Mechanical Behaviour

Abstract

There is a gap between mechanical measurement methods under laboratory conditions and those under real conditions of structural monitoring. This paper proposes a method that applies well-established computer vision developments to photomechanics in difficult conditions. It is therefore a technique that is flexible and versatile while maintaining satisfying accuracy. It consists in marker tracking using ArUco fiducial markers as measurement points. It allows locating markers in non-optimal conditions of camera orientation and position. A homography process was used to analyse pictures taken with a view angle. The accuracy of the method was estimated, especially in case of out-of-plane motions, and the impact of the view angle and of the distance between camera and markers on the location error was studied. The method was applied in creep tests to measure crack parameters as well as the transverse expansion of wooden beams. In the application example presented, it enabled to compute distances between markers with only 0.28% of relative error and hence to measure the crack parameters and the long-term shrinkage-swelling of the wooden beams. However, the impacts of brightness variations and camera parameters have not been estimated. This method is very promising when experimental conditions are variable and when multiple measurements are necessary.

Appendix: Homography

The aim of the homography is to project the coordinates in a given plane into another, as illustrated in Fig. 13 coming from [15]. In the left picture the photo is taken with a view angle and on the right picture it is projected onto the plane parallel to the wall.

Fig. 13

Left picture: view of a wall from an angle; right picture: transformed view of a wall to look like as if the camera were standing right in front of it

In this study, the goal is to go from the virtual image plane to the physical real world plane, where the real distance between two markers is defined by their coordinates. The detectMarkers function of OpenCV returns the position of the markers into the 2D image coordinates (i.e., in pixels).

In order to get real world coordinates (i.e., in meters), a homography must be computed:

$$\begin{aligned} \underline{X} = \omega \underline{\underline{H}}.\underline{x} \end{aligned}$$

Where:

• $\underline{X}=(X,Y,Z)$ are the real world coordinates;

• $\underline{x}=(x,y,1)$ are the homogeneous 2D image coordinates, related to the projective space. When passing from the Euclidean coordinates to the homogeneous coordinates, a dimension is added. This distance is fixed to 1 and allows to get mathematical coherence;

• $\underline{\underline{H}}$ is the $3 \times 3$ homography matrix:

  $$\begin{aligned} \underline{\underline{H}} = \left[ \begin{array}{lll} h_{11} &{} h_{12} &{} h_{13} \\ h_{21} &{} h_{22} &{} h_{23} \\ h_{31} &{} h_{32} &{} h_{33} \end{array}\right] \end{aligned}$$

  (A1)

• $\omega$ is a scale factor. In homogeneous coordinates, (x, y, 1) and $(\omega x, \omega y, \omega )$ represent the same point for $\omega$ a non-zero scalar. It may be fixed at $\omega = \frac{1}{h_{33}}$ and injected into $\underline{\underline{H}}$. In this case let’s note:

  $$\begin{aligned} \tilde{h_{ij}} = \frac{h_{ij}}{h_{33}}, \quad \quad (i, j) = ((1,2,3), (1,2,3)) \end{aligned}$$

The transformation needed to get the homogeneous 2D real-world coordinates is:

$$\begin{aligned} \underline{\hat{X}} = \left( \begin{array}{lll} X/Z \\ Y/Z \\ 1 \end{array}\right) = \left( \begin{array}{lll} \hat{x} \\ \hat{y} \\ 1 \end{array}\right) = \frac{\underline{X}}{Z} \end{aligned}$$

(A2)

It becomes:

$$\begin{aligned} \underline{\hat{X}}.Z = \tilde{\underline{\underline{H}}}.\underline{x} \end{aligned}$$

For a given point the equations are:

$$\begin{aligned} \left\{ \begin{array}{ll} \hat{x}.Z = \tilde{h_{11}}x + \tilde{h_{12}}y + \tilde{h_{13}} \\ \hat{y}.Z = \tilde{h_{21}}x + \tilde{h_{22}}y + \tilde{h_{23}} \\ Z = \tilde{h_{31}}x + \tilde{h_{32}}y + 1 \end{array} \right. \end{aligned}$$

Thus:

$$\begin{aligned} \left\{ \begin{array}{ll} \hat{x}.(\tilde{h_{31}}x + \tilde{h_{32}}y + 1) = \tilde{h_{11}}x + \tilde{h_{12}}y + \tilde{h_{13}} \\ \hat{y}.(\tilde{h_{31}}x + \tilde{h_{32}}y + 1) = \tilde{h_{21}}x + \tilde{h_{22}}y + \tilde{h_{23}} \\ \end{array} \right. \end{aligned}$$

By transforming $\underline{\underline{H}}$ into a vector, the following equation can be written:

$$\begin{aligned}{} & {} \left( \begin{array}{lllllllll} x &{} y &{} 1 &{} 0 &{} 0 &{} 0 &{} -x \hat{x} &{} - y \hat{x} &{} -1 \\ 0 &{} 0 &{} 0 &{} x &{} y &{} 1 &{} -x \hat{y} &{} - \hat{y} y &{} -1 \\ \end{array}\right) . (\tilde{h_{11}}, \tilde{h_{12}}, \tilde{h_{13}}, \tilde{h_{21}}, \tilde{h_{22}}, \nonumber \\{} & {} \qquad \tilde{h_{23}}, \tilde{h_{31}}, \tilde{h_{32}}, 1)^{T} = \left( \begin{array}{ll} 0 \\ 0 \end{array} \right) \end{aligned}$$

(A3)

Basically:

$$\begin{aligned} \underline{\underline{A}}.\tilde{\underline{H}} = \underline{0} \end{aligned}$$

Solving this problem, avoiding $\tilde{\underline{H}} = \underline{0}$, is the homography computation. It is a minimization problem as the solution is never perfect for images obtained experimentally. It is worth noting that 1 point in a picture with known real-world positions gives 2 equations, and the homography matrix consists of 8 unknowns. As a consequence, at least 4 points with known physical positions are necessary to compute the homography matrix. These 4 points may be the 4 corners of a single marker, or as in the case of this study 4 centers of 4 markers. There is at least two reasons for using 4 centers of 4 markers instead of 4 corners of a single marker:

• The centers are more precisely located than the corners;

• The projection of the pixels in the physical real world plane is globally better with 4 centers rather than 4 corners. Indeed, the projection with 4 corners is good mainly around that reference marker but not for remote pixels.

Obviously, 3 collinear points give a useless redundant equation, so that no more than two should be alined. If more than 4 points are available, the homography solution should gives a better minimization.

The homography used to get real world coordinates of a given picture is the same for all markers. Thus, they are all projected on the same plane. As a consequence, the homography has to be computed one time for each photo so that the distance between two markers is correctly computed even with a view angle involving a perspective.