A rigorous fastener inspection approach for high-speed railway from structured light sensors
Introduction
High-speed railways are booming in China, their total mileage had exceeded 20,000 km by the end of 2015 (Wang et al., 2016a). Their high speed (300–350 km/h) require high reliability and safety of the entire high-speed railway system. Fasteners play an important role in the high-speed railway track system, because they connect the rail track and the rail pad together, keeping the track gauge. Fasteners can reduce the vibration of the rail track (Zhang et al., 2016) and cut down the noise during the operation of the train (Köstli et al., 2008). Fastener failures, such as missing, partly broken or loose fastener, greatly influence the safety of trains (Esveld, 2001, Morales-Ivorra et al., 2016, Xiao et al., 2007, Zhou and Shen, 2013). Therefore, rail fasteners have to be inspected periodically by the railway management departments. Traditionally, rail inspection is conducted by trained workers who walk along the railway line to check the fasteners and other components. However, manual inspection is slow, labor-intensive, and dangerous. With the expansion of the high-speed rail lines, fastener inspection and maintenance face greater challenges. There is an urgent need for rail companies to replace manual fastener inspection with an automatic fastener inspection system.
In the last decade, many researchers have devoted their efforts to the development of fastener inspection methods based on two-dimensional vision, with most of the methods based on images (Babenko, 2009, Khan et al., 2014, Wang et al., 2016b). Marino et al. (2007) presented a real-time visual inspection system for automatic hexagonal-headed bolts. The system acquired images from a digital line-scan camera. Data were simultaneously preprocessed according to two discrete wavelet transforms and subsequently supplied to two multilayer perceptron neural classifiers. The system allowed an on-the-fly analysis of a video sequence that was acquired at 200 km/h. De Ruvo et al. (2009) presented a real-time vision system to detect the presence/absence of fasteners with a graphic processing unit (GPU). Xia et al. (2010) proposed a method to detect and recognize broken fasteners with complex ballast railway images based on the AdaBoost algorithm. Yang et al. (2011) presented an efficient method to detect fasteners on the basis of image processing and pattern recognition techniques, which could be used to detect the absence of fasteners on the track at a speed of up to 400 km/h. Feng et al. (2014) proposed an automatic visual inspection system for detecting partially worn and completely missing fasteners, using a probabilistic topic model. Their system got an overall precision of 98.7% and was possible to simultaneously model diverse types of fasteners with different orientations and illumination conditions using unlabeled data. Gibert et al. (2015) proposed a robust method for fastener inspection by using the histogram of oriented gradients features of fastener images and a combination of linear support vector machine (SVM) classifiers. The system described in the paper could inspect ties for missing or defective rail fastener problems with 98% precision. Some rail inspections methods are based on video. Resendiz et al. (2013) developed a machine vision system consisting of field-acquired video and subsequent analysis, to automatically detect irregularities and defects in wood-tie fasteners, rail anchors, and turnout components. All the methods described above are based on two-dimensional vision. Earlier methods could only detect the presence/absence of fasteners, whereas the recent methods can detect partly worn fasteners. Most two-dimensional vision (except for video) methods acquire vision data from the top to get the whole view of a fastener, making them unable to detect the loose fasteners.
Motivated by structural damage identification based on vibration, which is applied in structural health monitoring, Wei et al. (2017) proposed an automatic, remote-sensing fastener measurement system. In this system, several uniaxial accelerometers were fixed on the rail track, which was then excited by a hammer. The acceleration signals of the accelerometers were analyzed by wavelet packet analysis, through which the location of the damaged fasteners and the severity of looseness of the rail fasteners could be worked out. The sensor for this system needs to be pre-installed on the rail track thus seriously affecting the measurement speed and the convenience of the system.
Some researchers have focused on structured light methods. Structured light sensor is also called laser profilometer,1 laser profile sensor,2 or laser profile scanner (Kjellander and Rahayem, 2009). A structured light sensor consists of a structured light projector and an imaging sensor (a camera, for example). The structured light projector generates structured-light pattern on the target, which is then acquired by the camera. The measurement principle is based on triangulation calculation. When the shape of the target changes, the shape of the projected-light pattern changes as seen from the camera and accurate profiles of the target can be computed by various structured-light principles and algorithms (Geng, 2011). Compared to conventional point cloud acquisition technics such as Light Detection And Ranging (LiDAR), the scanning range of the structured light sensor is shorter (less than several meters) but the precision is higher (submillimeter). Several commercial or self-made structured light sensors have been applied in 3-D visualization (Seçil et al., 2014), component quality inspection (Burski et al., 2015), railway track measurement (Alippi et al., 2002, Liu et al., 2011, Sa et al., 2016), and even under water surface reconstruction (Sarafraz and Haus, 2016). Zhang et al. (2011) proposed a structured light method based on motion image (SLMMI) for moving object inspection. The combination of the SLMMI and a recognition method based on the neural network showed good performance in inspecting missing fastener components. Lorente et al. (2014) presented a structured-light-based system to evaluate rail gauge and detect missing rail fasteners. The railroad track is inspected by a range-based approach, while a 3-D iterative closest point (ICP) algorithm is applied with the manually predefined 3-D model of the fastener and previous knowledge of the expected location of the fastener. Aytekin et al. (2015) built a real-time railway fastener detection system to detect missing hexagonal-headed fasteners using structured light sensor. In that study, an extensive analysis of various methods, based on pixelwise and histogram similarities, was conducted on a specific railway route. All of the above methods based on the structured light sensor can only detect missing rail fasteners and none can identify partly worn or loose fasteners.
In this study, precise and dense 3-D point clouds for high-speed railway fasteners are obtained from commercial structured light sensors (Keyence LJ-V7000). With a decision tree classifier, not only the missing fasteners, but also partly worn or skewed fasteners can be detected. Since the metal clip is the crucial part of the fastener, region-growing method is used to segment it from the point cloud. Furthermore, a centerline extraction method for complex cylindrical surfaces is proposed to extract the centerline of the metal clip of a normal fastener from its point cloud. Subsequently, the looseness of the fastener is evaluated according to the extracted centerline of the metal clip. Experiments were conducted in high-speed railways near Wuhan, China, to collect data on three types of most commonly used fasteners. The experimental results demonstrated the accuracy and effectiveness of the proposed method. The proposed approach was also evaluated by experiments on the influence of different parameters and sparser input point cloud.
The rest of this paper is organized as follows: Section 2 provides an overview of the fastener system and evaluation of the point cloud accuracy. All the methodological steps of the proposed approach are explained in Section 3. Section 4 presents the experiment and results. Section 5 includes the discussion, while the conclusions are drawn in Section 6.
Section snippets
Fastener inspection system
There are many types of fasteners for high-speed railways. According to China railway design specifications, most of the existing and under-construction high-speed railways in China use three types of fasteners: Vossloh-300, WJ-8 and WJ-7, as shown in Fig. 1. These three types of fasteners have one characteristic in common: they all have a “”-shaped metal clip fixed next to the rail track, with a bolt in the middle. The bolt applies pressure to the metal clip to affix the rail track to the
Methodology
The overall steps of fastener inspection are illustrated in Fig. 6. Firstly, the fastener point cloud is extracted from the original point cloud. Subsequently, a decision tree classifier is used to classify the defect of the fastener. If there is something wrong with the fastener, its location and defect type are saved as the output result. If it is a normal fastener, the centerline of the fastener metal clip is extracted by the proposed centerline extraction method. The looseness of the
Experiment and results
In this section, several field tests were conducted on high-speed railways with different fastener types to evaluate the performance of the proposed fastener defect inspection approach. In the first part, the precision of the decision tree classifier was evaluated by comparing the results of the IRC fastener inspection system and the manual inspection. Subsequently, the accuracy of the centerline-based gap measurement method was evaluated by comparing the gap measured by the centerline
Discussion
The proposed decision tree classifier gets excellent results with approximately 99.8% precision realized in three types of most commonly used fasteners. In comparison, the image based fastener inspection method proposed by Feng et al. (2014) used the latent Dirichlet allocation (LDA) topic model to classify fasteners and achieved an overall precision of 98.7%. The fastener inspection method proposed by Gibert et al. (2015) is also based on image. Their method achieved 98% precision by using the
Conclusion
Fastener inspection is an important and challenging task in high-speed railway maintenance. This paper has proposed a rigorous fastener inspection approach for high-speed railway fasteners based on structured light sensor. Precise and dense point cloud of fasteners was obtained from structured light sensors. A fastener extraction method was used to extract the point cloud of fasteners. A decision tree classifier was proposed to check the specific state of the fastener, with which not only
Acknowledgment
Work described in this paper was jointly supported by the Fundamental Research Funds for the Central Universities under Grant No. 2042017kf0235 and National Key Research Program Grant No. 2016YFF0103502.
References (45)
- et al.
Automating land cover mapping of Scotland using expert system and knowledge integration methods
Remote Sens. Environ.
(2011) - et al.
CAMPINO—A skeletonization method for point cloud processing
ISPRS J. Photogram. Remote Sens.
(2008) - et al.
Simple and fast rail wear measurement method based on structured light
Opt. Lasers Eng.
(2011) - et al.
Texture augmented detection of macrophyte species using decision trees
ISPRS J. Photogram. Remote Sens.
(2013) - et al.
A structured light method for underwater surface reconstruction
ISPRS J. Photogram. Remote Sens.
(2016) - et al.
Oil spill feature selection and classification using decision tree forest on SAR image data
ISPRS J. Photogram. Remote Sens.
(2012) - et al.
Octree-based region growing for point cloud segmentation
ISPRS J. Photogram. Remote Sens.
(2015) - et al.
Full waveform inversion applied in defect investigation for ballastless undertrack structure of high-speed railway
Tunn. Undergr. Space Technol.
(2016) - et al.
An embedded system methodology for real-time analysis of railways track profile
- et al.
Railway fastener inspection by real-time machine vision
IEEE Transact. Syst., Man, Cybernet.: Syst.
(2015)
Visual Inspection of Railroad Tracks
The use of 2D/3D sensors for robotic manipulation for quality inspection tasks, solid state Phenomena
Trans. Tech. Publ.
A GPU-based vision system for real time detection of fastening elements in railway inspection
Curve and Surface Reconstruction: Algorithms With Mathematical Analysis
Modern Railway Track
Automatic fastener classification and defect detection in vision-based railway inspection systems
IEEE Trans. Instrum. Meas.
Structured-light 3D surface imaging: a tutorial
Adv. Opt. Photon.
Robust fastener detection for autonomous visual railway track inspection
Evaluation of center-line extraction algorithms in quantitative coronary angiography
IEEE Trans. Med. Imaging
Experimental and theoretical analysis of railway bridge noise reduction using resilient rail fasteners in Burgdorf, Switzerland
Automatic detection of defective rail anchors
Planar segmentation of data from a laser profile scanner mounted on an industrial robot
Int. J. Adv. Manuf. Technol.
Cited by (46)
Nano-fiber based self-powered flexible vibration sensor for rail fasteners tightness safety detection
2022, Nano EnergyCitation Excerpt :However, manual detection method has low efficiency, high labor intensity and high missed detection rate [4]. Computer vision detection method has high cost and poor adaptive performance [5]. These limitations hinder their widespread application.
Frost heave force analysis of pre-embedded dowel in the high-speed railway ballastless track
2022, Construction and Building MaterialsCitation Excerpt :With the rapid development of high-speed railway in the world, the ballastless track is widely used because of its remarkable advantages such as good stability, durability, smoothness and easy to maintain track geometry [1–5]. The fastener system is one of the core components of the track structure, and its main function is to connect the rail and the ballast bed into a stable and overall structure which can ensure the safety of the train operation [6–8]. As an important part of the fastener system [9–10], some pre-embedded dowels are found to be damaged or failed due to the train load and the complex natural environment.
A real-time railway fastener inspection method using the lightweight depth estimation network
2022, Measurement: Journal of the International Measurement ConfederationCitation Excerpt :The fastener area is only a small part of the raw data collected by the sensor, so the first task is to locate and extract it. Prior work [8,9] uses the sliding window method to locate fastener areas from point clouds based on the height information. However, in our method, the height information is absent in the test stage, and thus we need a fast and robust object detection method based on images.
A universal method for the calibration of swing-scanning line structured light measurement system
2021, OptikCitation Excerpt :Based on the laser triangulation principle, the profile information of an object can be computed by use of the perturbed stripe image, the camera intrinsic parameters and the equation of laser plane [1]. Due to its advantages of non-contact, simple construction, low cost and easy application, it has gained many applications in geometrical measurement [2,3], quality evaluation [4] and industrial inspection [5]. For traditional LSLS, the laser line projector has a fixed relative position with the camera.
Regression Method for Rail Fastener Tightness Based on Center-Line Projection Distance Feature and Neural Network
2024, Chinese Journal of Mechanical Engineering (English Edition)Railway Fastener Anomaly Detection via Multisensor Fusion and Self-Driven Loss Reweighting
2024, IEEE Sensors Journal