Vision-Based Structural Identification Using an Enhanced Phase-Based Method ^†

Abstract

Operational modal analysis is based on data collected using a network of sensors installed on a monitored structure to measure its response to the external stimuli. As the instrumentation can be costly, sensors are placed at a limited number of locations where damage-sensitive features can hopefully be sensed. Hence, the actual ability to detect a shift from the undamaged structural state in real time might be detrimentally affected. Non-contact measurement methods relying on, e.g., digital video cameras, which have gained interest in recent years, can instead provide high-resolution and diffused measurements/information. In this study, moving from videos of a vibrating structure, a shift in its dynamic response was assessed. By means of a phase-based optical flow methodology, a linear correlation between the phase and the structural motion was customarily assumed using, e.g., the Gabor filter. Since such a correlation does not result in being always linear, linearization is necessary for all the frames. By using the blind source separation method, mode shapes and vibration frequencies were finally obtained. The performance of the proposed method is investigated to verify the accuracy in extracting the dynamic features of the considered structure using the proposed method.

1. Introduction

Modal analysis is a vital tool used to identify the dynamic behavior of structures in terms of natural frequencies, mode shapes, and damping ratios [1,2,3]. By understanding how the structures respond to the external forces, it can be ensured that they withstand such loads as well as the relevant environmental conditions [4]. Modal analysis serves as a crucial tool used in model order reduction techniques [5,6] and structural health monitoring (SHM) [7]. In recent times, the use of vision-based measurements has emerged as a highly effective method for full-field identification [8,9], damage detection [10], model updating [11], and response measurement [12,13]. This innovative approach exploits the data from images to gain insights into structural behavior. Besides high-resolution response measurements, it avoids the additional weight linked to the sensing system and reduces the cost of purchasing, installing, and maintaining sensors.

Optical flow estimation is a method used in computer vision that tracks the movement of pixels between consecutive frames of a video. Fleet and Jepson [14] demonstrated that the local phase of an image, obtained through quadratic filters [15], represents motion more robustly than the intensity. Phase-based motion magnification, introduced by Wadhwa et al. [16], involves the amplification of subtle motions in a video sequence by focusing on phase information. By means of this technique, Yang et al. [17] obtained mode shapes from videos of vibrating structures. These mode shapes were subsequently employed in [18] to identify and locate damage (also refer to [19]). Southwick et al. [20] expanded this approach to extract 3D volumetric motions. Luo et al. [21] introduced a novel image-processing technique that addresses the challenge vision sensors face in outdoor structural displacement monitoring, as the conventional approach is susceptible to noise and limited in its measurement range. To enhance accuracy, Cai et al. [22] addressed the limitations of phase-based estimation and proposed a novel multi-view measurement method. Miao et al. [23], by optimizing the Gabor filter parameters, introduced a robust phase-based displacement measurement technique to capture vibration responses.

In this paper, we explore the impact of a specific filter parameter on the filtering response. As shown in the results, it is crucial to recognize that not all the pixels in an image are suitable for motion detection or identification processes; the selection of the appropriate one(s) is a critical concern. Via a defined criterion, regions are identified where the linear correlation between phase and motion is not established. This detection process enhances the accuracy of motion estimation. The phase-based displacement measurement was applied to a real case test, and phase-based identification, incorporating Independent Component Analysis (ICA) blind-source separation, was performed on videos of a vibrating structure.

2. Motion and Phase Relation

In one-dimensional signal analysis, phase delay is related to how signals evolve over time; the same concept can be applied to two-dimensional signals like images. When a feature in an image undergoes spatial movement, like a translation or a rotation, this process leads to changes in the local phase of the pixels. This shift in phase is directly related to the extent and direction of the movement.

Fleet and Jepson [14] explored the connection between local phase difference and motion. By tracking the constant phase contours in successive frames, a motion field can be obtained. Assuming that the intensity of the first frame at time t₀ in a pixel with coordinates $X\left(x,y\right)$ is $I\left(X\right)$ if it becomes displaced by $∆\left(X,t\right)$, the intensity profile of the next frame at time t₁ is $I(X+∆\left(X,t\right))$. To extract the local phase, it is necessary to use filters like the Gabor filter.

The response of the Gabor filter is a complex valued function that can be expressed as

$$R\left(X,t\right)=\rho \left(X,t\right)\times e^{i\varphi \left(X,t\right)}$$

(1)

where $\rho \left(X,t\right)$ and $\varphi \left(X,t\right)$ represent its amplitude and phase component, given by

$$\begin{gathered} \rho \left(X,t\right)=\left|R\left(X,t\right)\right|=\sqrt{{\mathbb{R}\mathbb{e}\left[R\left(X,t\right)\right]}^2+{\mathbb{I}\mathbb{m}\left[R\left(X,t\right)\right]}^2} \\ \varphi \left(X,t\right)=\arg \left[R\left(X,t\right)\right] \end{gathered}$$

(2)

The tracking of continuous changes in the phase contours over time, providing a reliable approximation of the motion field. In simple terms, points represented by $X\left(x,y\right)$ on these contours maintain a constant value of $\varphi \left(X,t\right)=c$. The displacement in a direction $\theta$ is then derived from the movement of local phase contours.

3. Unreliable Phase Detection

Motion estimation from videos is characterized by a significant challenge: it is not feasible to extract genuine motion data from every pixel. This issue arises due to various factors, including fluctuations in lighting conditions, Gabor filter parameters, varying object scales within an image, and the presence of noise.

To provide an illustration of this problem, Figure 1a displays an image of a chimney located at Politecnico di Milano, as captured using a standard mobile phone camera. Images were then processed to create a simulated displacement scenario (ten pixels).

By applying the Gabor filter with two different scales, the phase and amplitude obtained from suitable pixels were selected for displacement measurement. When the phase varies between values of π and −π, jumps (phase wrapping) occur, but they should not be considered as unstable regions. A simple way of unwrapping the phase was adopted in this study: when the phase difference between two frames at a point is greater than π, 2π is added or subtracted.

In Figure 1d, displacement measurements with three colors are shown: the green region stands for measurements characterized by an error smaller than 5%, while the red region features measurement errors larger than 5%; the cyan region is instead related to an unstable phase. As shown in the figure, the area representing accurate measurements increases in size at a higher scale, but finer details are lost; for instance, the upper-left part of the tower is not captured by the selected pixels at the higher scale.

The phase-based motion estimation methodology is based on the linear correlation between the phase and the structural motion. However, phase nonlinearity stands out as the primary cause of inaccurate motion estimation. By detecting the region in which the phase contours are not likely to provide reliable information about the motion, the measurement error can be reduced. According to [24], two constraints may be needed to detect the points with such a behavior. This approach was adopted here, with different bounds on the output of the applied Gabor filter tuned to one scale. The first one is the frequency constraint: the instantaneous frequency of the filter output, which is the spatial gradient of the phase ${\varphi }_x\left(X,t\right)$, was constrained such that it would be close to the value at which the filter was tuned:

$$\left|{\varphi }_x\left(X,t\right)-k\right|<\tau$$

(3)

The second constraint was instead based on the amplitude: its derivative ${\rho }_x\left(X,t\right)$ was constrained such that it would be small, according to:

$$\left|{\rho }_x\left(X,t\right)\right|/\rho \left(X,t\right)<\tau$$

(4)

To determine the impact of these constraints, the displacement field between two frames in Figure 1a were computed at four scales. Pixels were selected using a simple threshold on the amplitude using Equations (3) and (4). Figure 2 shows that by employing these thresholds with $\tau =0.05$, the cyan regions are completely avoided, and the red region is reduced in comparison with that linked to the phase selection solely based on the amplitude.

4. Full-Field Identification

An investigation of the performance of pixel selection based on the proposed criteria is provided in this section, with the aim of identifying full-field modal shapes and vibration frequencies. A model of a ruler subject to free vibrations was captured in a video using MATLAB. Non-linearity in the phase was introduced to account for the lack of alignment between the filter orientation and the edge of the body. The process of identification is depicted in Figure 3.

By defining the appropriate pixels on the basis of the discussion in Section 3, a matrix was created to represent the time series of the selected phases across all frames. By employing PCA, the dimensionality of this matrix was reduced to the number of excited modes, which turned out to be three in the present case. In general cases, this number of modes is set on the basis of the eigenvalues of the original matrix: to avoid issues related to noise, only principal components linked to eigenvalues larger than 1% of the maximum value were retained in the model. Afterward, by way of ICA [25], the frequencies of the vibrations and the corresponding mode shapes were obtained.

In Figure 4a, the identified frequencies (2.99, 18.45, and 51.87) are compared with their corresponding real values (2.93, 18.63, and 51.77) to demonstrate the remarkable accuracy obtained, amounting to 98%, 99.04%, and 99.8%, respectively. Figure 4b shows the identified mode shapes, which are compared to the real ones using the modal assurance criterion, revealing similarities of 99.1%, 96.7%, and 98.2%, respectively. In Figure 5, it is further demonstrated that the selected regions all have consistent representations of the mode shapes.

5. Conclusions

In this study, the impact of the bandwidth of the Gabor filter on the phase extracted from images of a video of structural vibrations was investigated. By enlarging the value of this parameter, the level of detail under examination is decreased; alternatively, by decreasing it, the linear relationship assumption between phase and motion is violated. Choosing appropriate pixels for the analysis therefore poses challenges in image processing. By imposing specific conditions on the phase and amplitude of the filtered images, artificial displacement in a real structure was examined. It was observed that these conditions allowed for the identification of unsuitable phase areas in order to attain more-accurate solutions. Moreover, the selected phases were exploited for blind modal identification after dimensionality reduction through PCA and the application of ICA. The identified frequencies and modal shapes were shown to accurately match the real ones.

In this investigation, pixel selection was performed exclusively using the first frame of the video. However, considering the potential effects of non-linearity, this choice might differ across different frames. The objective in future research will be to identify these locations across all frames, remove them, and subsequently provide accurate estimations using learning-based methods.