Tool wear condition monitoring based on multi-sensor integration and deep residual convolution network

In this paper a tool wear condition monitoring system is proposed based on multi-sensor integration and deep residual convolution network. The proposed tool wear monitoring system is designed as part of an online tool wear management system attached to the CNC machine. It is supposed to collect indirect data about tool wear during the machining process, identify the tool condition and report to the controller for tool management. Because the cutting tools are difficult to measure directly in machining, a combination of sensors are used to collect multi-source data related to the cutting, and deep learning methods are employed to estimate the tool condition indirectly.

Figure 1 shows the framework of the proposed tool wear monitoring system. It is mainly comprised of three functional modules namely signal acquisition, feature extraction, and tool condition identification.

• (1)  
  The signal acquisition module. This module is for data collection and preprocessing. In order to minimize the interference in the collected data, a number of sensors are employed and multiple sets of data from the same sensor are used. Combined data sets may reduce the system's sensitivity to noise and temporary faults. Multiple independent sensor data sources help improve accuracy by eliminating ambiguities in the data. In this system, multi-sensor fusion technology is adopted for data acquisition. The multi-sensor signals complement each other in tool wear perception, with the advantages of strong anti-interference ability and wide coverage of information. In order to ensure the multi-source data collected during machining carry effective information about the cutting tool condition, a selection of sensors (acoustic emission sensors, sound sensors and acceleration sensors in this case) should be installed near the cutting area systematically. The data signals collected by the multi-sensors will be amplified first, with the initial and final signal segments being cut out, then transferred to the database to form the initial data. This makes it convenient for the feature extraction module to perform the subsequent data processing and analysis tasks.
• (2)  
  Feature extraction module. This module is the main part of the system in terms of computation and machine learning. It is responsible for two important tasks: signal data fusion imaging and feature extraction. In order to comprehensively analyze the condition of cutting tools, angular matrix imaging of signal sequence is used to map the collected multi-sensor signals into two-dimensional images containing time series features, and the signals are transformed into image input forms that are easier to be recognized and processed by the deep learning module, which will be discussed in detail in section 2.2. The second task is feature extraction. It uses the obtained images first as input for two-layer convolution network to extract shallow features, and then as input for deep residual network to extract deep features of the data through deep learning. For better extraction, the deep residual network takes the advantage of attention mechanism by combining with a Convolutional Block Attention Module (CBAM). Specifically, CBAM block is formed by serial connection of channel attention mechanism module unit and spatial attention mechanism module. On this basis, the residual unit connects the CBAM input and output through layer hopping connection, which will be discussed in detail in section 2.3. The application of attention mechanism is CBAM block of comprehensive channel and spatial domain, which will be addressed in detail in sections 2.2 and 2.3.
• (3)  
  Tool wear state identification module. The main function of this module is to determine the wear state of the tool. Since the output of the feature extraction module is not yet the desired final decision, it is necessary to combine the high-dimensional features extracted from the previous deep residual network with local important features, and send the final feature vector to the Softmax classifier through global pooling, so as to realize the mapping of data features and sample labels, and then realize the classification of tool wear states. At the same time, the classification results are stored in the tool management database of the computer, and the CNC system can obtain corresponding feedback instructions according to the classification results. For example, when the tool is detected to be in the rapid wear state, the cutting parameters may be changed to extend the tool life or a warning be issued in advance. When the tool is detected to be in the severe wear state, the tool replacement warning will be issued or start a tool change operation.

Zoom In Zoom Out Reset image size

The angular matrix imaging method reported in the literature [18] is utilized in this paper to fuse sensor signals of multiple dimensions at the data level. Taking for example the signal imaging measured by piezoelectric acceleration sensor, the collected signal is a point set of $({t}_{i},{x}_{i}),$ where ${x}_{i}$ represents the acceleration value measured at the corresponding time point (obtained by signal amplifier), and ${t}_{i}$ represents the timestamp. Assume $({r}_{i},{{\rm{\Phi }}}_{i})$ is the polar coordinates for each point $({t}_{i},{x}_{i}).$ After standardizing the acceleration value at the ${t}_{i}$ time point and then the angle ${{\rm{\Phi }}}_{i}$ in the form of corresponding polar coordinate representation is obtained as arccosine of the normalized value of ${x}_{i},$ and ${r}_{i}$ is obtained by reducing the $t\,$unit. By doing this in turn for all the signal points on the time axis, the polar coordinate transformation of a sensor signal is completed. After transforming the scaled time series into polar coordinates, the time relationship between different time points is identified by calculating the cosine value of each time point. The specific imaging process is shown in table 1.

Table 1. Angular matrix imaging process.

S. no.	The content of process
1.	• The signal time series $X$={${x}_{1},$ ${x}_{2},$..., ${x}_{i},$..., ${x}_{n}$} is normalized and reduced to the value ${x}_{i}$ of ${t}_{i}$ at each time point, resulting in $\tilde{X}$={$\tilde{{x}_{1}},$ $\tilde{{x}_{2}},$..., $\tilde{{x}_{i}},$..., $\tilde{{x}_{n}}$}, where $-1\lt \tilde{{x}_{i}}\lt 1.$ Each ${t}_{i}$ is reduced by unit to get ${r}_{i},$ where ${r}_{i}={t}_{i}/n$ and $n$ is the regularization factor, and the polar coordinate transformation is completed.
2.	• For each element of $\tilde{X}\,,\,$the corresponding ${{\rm{\Phi }}}_{i}$ is calculated as ${{\rm{\Phi }}}_{i}=ar\,\cos \left(\tilde{{x}_{i}}\right);$ 则$\tilde{X}=\left\{\cos \,{{\rm{\Phi }}}_{1},\,\cos \,{{\rm{\Phi }}}_{2},\,\cos \,{{\rm{\Phi }}}_{3},\ldots ,\,\cos \,{{\rm{\Phi }}}_{n}\right\}$
3.	• Note ${\tilde{X}}^{\text{'}}=\left\{\sin \,{{\rm{\Phi }}}_{1},\,\sin \,{{\rm{\Phi }}}_{2},\,\sin \,{{\rm{\Phi }}}_{3},\ldots ,\,\sin \,{{\rm{\Phi }}}_{n}\right\}$
4.	• Let $\tilde{X}\,and\,{\tilde{X}}^{\text{'}}\,$make Kronecker products with their transpose matrices respectively, and then make the difference to get a symmetric matrix ${T}_{1},$ which contains the time information in the data
5.	• All elements of the lower triangle of matrix ${T}_{1}\,$are set to 0 to obtain the upper triangle matrix ${{T}_{1}}^{\text{'}},$ that is, the two-dimensional matrixization of the signal data of a single sensor is completed.
6.	• Repeat Step1-Step5 operation to obtain ${T}_{2}\,$for another sensor signal collected, and transpose ${T}_{2}$ to obtain matrix ${T}_{2}^{\text{'}}\,.$ By adding ${T}_{1}^{\text{'}}\,and\,{T}_{2}^{\text{'}},$ the two-dimensional matrix containing two sensor data is completed, and one channel image is obtained.

Following these steps, the multi-sensor data are prepared in pairwise combinations to form two-dimensional matrices by means of imaging, these data are then wrapped as one channel image. When there are three single-channel images, they can be combined into an RGB three-channel image as the data input of the subsequent module. Based on this method, the inherent time relationship of each sensor signal can be retained to achieve effective integration of sensor data in data layer. The visual effect of sensor data fusion is intuitively displayed by feature pictures.

The essence of the attention mechanism is to focus on useful information and avoid interference. By automatically adjusting the parameter configuration after system training, it can extract useful information like human vision, reduce the interference of useless information, and improve the extraction efficiency. In this paper, the attention mechanism functions by means of a channel attention unit and a spatial attention unit together, which are combined in order in every block. Since the attention mechanism starts in two dimensions, which can complement each other, making the network pay attention to important features and suppress unnecessary ones. In order to increase the ability to mine deep features from the data, a residual block structure is added on the basis of attention mechanism, which forms an attention-based residual unit. The depth of the network may be improved by increasing the number of this unit the residual unit within the network.

Figure 2 shows the structure of a single attention-based residual unit. $F$ represents the feature map obtained by the convolution operation of the previous residual unit based on attention mechanism, it can be seen that feature map F has three-dimensional features, and H, W and C represent the size in three directions respectively; $F\text{'}$ represents the feature map after channel attention extraction; $F\text{'}\text{'}$ represents the feature map after spatial attention extraction; ${M}_{c}$ represents the operation of attention extraction in the channel dimension; ${M}_{s}\,$represents the operation of attention extraction in the spatial dimension.

Zoom In Zoom Out Reset image size

As shown in figure 2, the channel attention mechanism works as follows: the feature map $F$ is taken with dimension $H\,\ast \,W\,\ast \,C$ as input, and then go through global maximum pooling and global average pooling based on width and height to obtain two feature maps. Next, they are sent to a two-layer artificial neural network (MLP) respectively, and the features output by MLP are added based on element summation. After sigmoid activation, the final channel attention feature is generated, that is, ${M}_{c}(F).$ Finally, perform element-wise multiplication between ${M}_{c}(F)$ and input feature map $F$ to generate input feature ${F}^{\text{'}}$ required by spatial attention module.

The spatial attention mechanism is realized as follows: the feature map ${F}^{\text{'}}$ outputted from the channel attention module is used as the input feature map. First, do a global maximum pooling and a global average pooling based on channel to get two $H\,\ast \,W\,\ast \,1$ feature maps, and then do channel splicing based on channel dimension. After that, through a 7* 7 convolution operation, the dimension is reduced to one $H\,\ast \,W\,\ast \,1$ single-channel feature map. And then the sigmoid function generates a spatial attention feature ${M}_{s}({F}^{\text{'}}).$ Finally, the feature is multiplied with the input feature ${F}^{\text{'}}$ of the module to obtain the final feature $F\text{'}\text{'}.$

As mentioned above, the proposed multi-sensor TCM system is supposed to work with CNC milling machine tools. Major parameters of the CNC machine tool related to the machining test are shown in table 2. To collect tool condition data and train the system, the ACE-V500 vertical machining center is utilized, with sensors including an acceleration sensor, acoustic emission sensor and microphone sensor mounted as shown in figure 3. The workpiece is a hexahedral block (200 mm * 100 mm * 60 mm) made of 45#steel (GB/T699-1999). A high-speed data collector (TranAX 4.1) is selected for data acquisition in the experiment. The sampling frequency is set to 10 kHz for the experiment.

Zoom In Zoom Out Reset image size

It is worth mentioning that the indirect method of measurement is more economical and practical for online monitoring of tool conditions, as long as the sensors used in a TCM system fit in well with the whole machining system. At present, the commonly used sensor signals include cutting force, acceleration, acoustic emission, current and sound. Among them, the cutting force signal has higher sensitivity and faster response, but the installation of a dynamometer often requires retrofitting of the machine tool. Balance the accuracy, cost, and installation, the acceleration sensor, acoustic emission sensor and microphone sensor are selected for data collection purpose in this paper. Table 3 lists the detailed information of the selected sensors. In addition, an MV-CE060-10UM optical sensor is used in the experiment for measurement of the tool wear.

The sensors for indirect measurement of the cutting tool are mainly positioned on the workpiece, spindle, fixture or workbench. The closer they are to the cutting area in the actual data acquisition process, the higher the signal intensity and quality of the collected data. Therefore, in this paper, the acceleration sensor is attached to the workpiece with strong glue. In addition, considering that the material to be processed in this experiment is non-magnetic, the AE sensor with magnetic holder is arranged on the fixture close to the cutting area. The signal from the microphone sensor is transmitted through sound waves. In order to collect the highest quality sound signal, the microphone sensor is placed close to the cutting area.

Before application of the proposed system, it is necessary to collect data and use them to train the network for wear state recognition in machining. To that end, cutting experiments are carried out following a number of steps.

Step 1: Install a new cutting tool and set up the workpiece. The tool is made of cemented carbide, with MT-TiCN + Al2O3 coating.

Step 2: Calibrate the microphone sensor for data acquisition, with its sampling frequency set to 10 kHz.

Step 3: Install acceleration and acoustic emission sensors on the surface of the workpiece, and install the microphone sensor as close as possible to the cutting tool.

Step 4: Set machining parameters are set according to table 4.

Step 5: Start cutting the workpiece and the signals are acquired during the period from the moment when the cutting tool cuts in the workpiece to the end of the cutting. After each cutting operation, the spindle of the machine tool is assigned to a fixed starting position, and then adjust the rotation angle of the cutter holder and use the optical sensor to take high-definition pictures of the rear face of cutter, and record the wear amount of the tool.

Repeat step 5 until the tool is worn out. In actual cutting, when the wear amount on the rear face of the cutter reaches 0.2 mm, the cutter is replaced. The complete life of such a tool is recorded, and then another tool is used for cutting. In this way, two tools are measured, and the data obtained are recorded as T1 and T2 respectively for later use.

For the measurement method of tool wear in step 5, the following values are mentioned: if the tool is removed at the end of each cutting cycle and the wear value is measured manually, this method is too time-consuming and does not guarantee the continuity of the experiment. Therefore, we improve this method. At the beginning of the cutting cycle, that is two or three times cutting cycle, the optical sensor takes the high-definition picture of the back of the tool and takes the tool out. The wear of the back of the tool in the high-definition picture is taken as the measurement standard, i.e. the 1 mm standard, and then the actual wear is measured with the microscope, i.e. the standard value. As shown in figure 4, by setting the light angle, the brightness of the wear area on the back of the tool becomes more pronounced. By measuring the width of the gap between the two red lines marked in the photograph and multiplying by the standard, the actual wear amount can be obtained. In this way, the actual wear value of the tool can be obtained for all the cutting cycles.

Zoom In Zoom Out Reset image size

As pointed in the literature [19], tool wear roughly follows a certain pattern, from initial wear through normal wear to severe wear. In the normal wear stage, the rough surface of the cutting edge is worn off, the working pressure becomes even, and the tool wear is relatively slow. The cutting tool works effectively in this stage. Then, the friction and cutting temperature between the tool and the workpiece would increase, and consequently, the tool wear would speed up significantly. After that, the tool condition gradually evolves into severe wear condition, under which the tool is unable to guarantee actual machining requirements and would even lead to tool crack or machine halt. Therefore, the TCM system needs to effectively identify the tool condition before it enters the severe wear state.

Figure 5 shows the wear curves in full lifecycle of the cutting tools (T1 and T2), which are obtained based on the wear amounts measured during the cutting experiment. It shows that the wear amount increases quickly in the initial wear stage, which lasts only a short time, then the wear develops steadily until it approaching 0.02 mm, and finally the tool is worn out and changed after about 64 consecutive cutting cycles. As the cutting tools in severe wear condition are prone to cause severe problems and the tool condition usually degenerate quickly just before it enters the severe wear state, a rapid wear state is inserted, in order for quick recognition of the tool condition before in enters the severe state. Thus, the tool wear process is divided into four states: initial wear, normal wear, rapid wear and severe wear. Figure 6 shows the worn rear face of a cutting tool (T1) measured in different wear states during the experiment.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

It is worth noting that the time series of each cutting sample collected is up to 200 000. In order to reduce the adverse effect of class imbalance on the performance of feature extraction module, under-sampling methods are also used to balance the data between the wear types, as indicated in table 5.

After data collection and data preprocessing, the TCM system is trained using the collected data. In order for stable signal processing and feature extraction, certain computer hardware is used including a Core i7700 processor with the clock frequency of 3.6 GHz, a DDR4 64GB memory and a GeForce RTX 2080ti GPU. The software adopts a deep learning framework with Keras as the front-end and Tensorflow as the back-end.

Hyperparameters also play an important role in guaranteeing the performance of the deep learning model. In this work, the grid search method is adopted to preliminarily select the hyperparameters for deep learning model. Based on previous work [17], the Learning rate, Epoch, and Batch Size are respectively set to 0.001, 100 and 32; and the optimizer works in Adam mode. Authors are referred to reference [20] for detailed illustration on hyperparameters selection.

In order to verify the trained monitoring system, a cutting experiment under similar cutting conditions with a new cutting tool (T3) is conducted. The same sensors are used to collect real-time signals for the trained system to recognize the tool wear state. As in the data-collection experiment, the cutting involves quite a number of cycles, but it does not need to stop for measurement after each cycle. The cutting is supposed to continue until the monitoring system reports a sever state. However, in order to check the predicted result against the real wear amount, the cutting operation is paused 10 times between cutting cycles for measurement. Four each time, the tool wear amount and the multi-sensor data are recorded, and the whole data set is marked as T3. Then the data set T3 is used to verify the monitoring system.

Based on the tool state classification discussed in section 4.1, a number of sampling points are selected in the verification experiment as shown in table 6, each referred to as a test. The sequential number of the cutting cycle is also listed in the table, together with the actually measured wear amounts, and the wear condition both judged by measurement and predicted by the monitoring system.

Figure 7 shows a few sections of the acuired signals and illustrates their pre-processing for the experiment, taking a small part of the sampled data in Test 9 as an example. As shown in the figure, five channel sensor signals are obtained including the acceleration signals in the X, Y and Z directions, the acoustic emission signal and the microphone signal. Since the time series collected by each sensor is up to 200,000, a 200,000 × 200,000 image would be obtained through Angle matrix imaging. In order to reduce the amount of computation, a concise subsequence is obviously preferred. On the other hand, in the light of the spindle speed (400 rpm) and the data sampling frequency (10 kHz) in the experiment, the length of the time series corresponding to a rotation of the tool should be greater than 1500. Thus, the middle section of each sample is selected only and a subderies with a length of 2000 is obtained to represent the sample.

Zoom In Zoom Out Reset image size

In order to smoothe the subseries, the piecewise aggregation approximation (PAA) [21] is used to compress their length to 256, and then the subseries are mapped to single channel images by means of the angular matrix imaging method. These pictures are combined as shown in the figure, and finally a three channel image (256 * 256 * 3) is formed. The obtained image is used as the initial input of the system's deep residual network, and the prediction is, in this instance, rapid wear.

Table 6 shows the detailed information of the tests and results. It shows the accuracy of wear state recognition of the 10 samples is 90%, with one false estimation just at the beginning of normal stage. This indicates that the proposed system pays more attention to the recognition of rapid and severe wear condtion than that of initial znd normal wear condition. Figure 8 shows how the tool wear deceped during the machining process. It indicates that the wear curve of this cutting tool also conforms to the general pattern of tool wear. As a whole, the proposed system demonstrate fairly good performance in the experiment, thanks to the integration of multi-sensor data and the deep residual network to learn the potential features of sensor information.

Zoom In Zoom Out Reset image size