Skip to main content
SpringerLink
Log in

Research and prospect of welding monitoring technology based on machine vision

  • Critical Review
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Welding monitoring technology based on machine vision has been widely researched in academic and industry, especially in the background of Industry 4.0, in that it can contribute to welding quality and productivity improvement. This paper outlines the technical points of welding status monitoring based on machine vision, including hardware and software. First of all, in the hardware part, the active and passive vision systems are briefly introduced, as well as the key steps in experimental deployment, such as the configuration of optical sensors and optical filters based on different detection objects. Secondly, some related image processing techniques in welding monitoring are also comprehensively reviewed. Additionally, the observed objects and their morphological characteristics of vision-based welding process monitoring are enumerated. On this basis, a series of intelligent models as well as optimization methods for recognition and classification in visual monitoring are considered in detail. Finally, potential research challenges and open research issues of welding visual monitoring are discussed to present an insight into future research opportunities. The main purpose of this paper is to provide a reference source for the researchers involved in intelligent robot welding.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4.
Fig. 5
Fig. 6
Fig. 7
Fig. 8.
Fig. 9
Fig. 10.
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Pan Q, Mizutani M, Kawahito Y, Katayama S (2016) High power disk laser-metal active gas arc hybrid welding of thick high tensile strength steel plates. J Laser Appl 28(1):012004

    Article  Google Scholar 

  2. Stavridis J, Papacharalampopoulos A, Stavropoulos P (2018) Quality assessment in laser welding: a critical review. Int J Adv Manuf Tech 94(1):1–23

    Google Scholar 

  3. Maeda K, Suzuki R, Suga T, Kawahito Y (2020) Investigating delayed cracking behaviour in laser welds of high strength steel sheets using an X-ray transmission in-situ observation system. Sci Technol Weld Joi 25(5):377–382

    Article  Google Scholar 

  4. W Y, B L, C C, M L, X Z, W F, G JP, J H, F Y (2020) Microstructure and mechanical properties of stainless steel clad plate welding joints by different welding processes. Sci Technol Weld Joi 25(7):571–580

    Article  Google Scholar 

  5. You D, Gao X, Katayama S (2014) Review of laser welding monitoring. Sci Technol Weld Joi 19(3):181–201

    Article  Google Scholar 

  6. You D, Gao X, Katayama S (2014) Multisensor fusion system for monitoring high-power disk laser welding using support vector machine. IEEE T Ind Inform 10(2):1285–1295

    Article  Google Scholar 

  7. Webster PJL, Wright LG, Ji Y, Galbraith CM, Kinross AW, Van Vlack C, Fraser JM (2014) Automatic laser welding and milling with in situ inline coherent imaging. Opt Lett 39(21):6217–6220

    Article  Google Scholar 

  8. Kong F, Kovacevic R (2010) 3D finite element modeling of the thermally induced residual stress in the hybrid laser/arc welding of lap joint. J Mater Process Tech 210(6-7):941–950

    Article  Google Scholar 

  9. Zhang F, Liu S, Liu F, Zhang H (2019) Stability evaluation of laser-MAG hybrid welding process. Opt Laser Technol 116:284–292

    Article  Google Scholar 

  10. Pan Q, Mizutani M, Kawahito Y, Katayama S (2016) Effect of shielding gas on laser-MAG arc hybrid welding results of thick high-tensile-strength steel plates. Weld World 60(4):653–664

    Article  Google Scholar 

  11. Mirapeix J, Ruiz-Lombera R, Valdiande JJ, Rodriguez-Cobo L, Anabitarte F, Cobo A (2011) Defect detection with CCD-spectrometer and photodiode-based arc-welding monitoring systems. J Mater Process Tech 211(12):2132–2139

    Article  Google Scholar 

  12. Chen Y, Chen B, Yao Y, Tan C, Feng J (2019) A spectroscopic method based on support vector machine and artificial neural network for fiber laser welding defects detection and classification. Ndt&e Int 108:102176

    Article  Google Scholar 

  13. Liu L, Shi J, Hou Z, Song G (2018) Effect of distance between the heat sources on the molten pool stability and burn-through during the pulse laser-GTA hybrid welding process. J Manuf Process 34:697–705

    Article  Google Scholar 

  14. Zhang C, Gao M, Zeng X (2019) Influences of synergy effect between laser and arc on laser-arc hybrid welding of aluminum alloys. Opt Laser Technol 120:105766

    Article  Google Scholar 

  15. Lober R, Mazumder J (2007) Spectroscopic diagnostics of plasma during laser processing of aluminium. Journal of Physics D-Applied Physics 40(19):5917–5923

    Article  Google Scholar 

  16. Sibillano T, Rizzi D, Ancona A, Saludes-Rodil S, Rodriguez-Nieto J, Chmelickova H, Sebestova H (2012) Spectroscopic monitoring of penetration depth in CO2 Nd:YAG and fiber laser welding processes. J Mater Process Tech 212(4):910–916

    Article  Google Scholar 

  17. Chen B, Chen Z, Cheng H, Tan C, Feng J (2018) Spectral analysis of the plasma emission during laser welding of galvanized steel with fiber laser. Transactions on Intelligent Welding Manufacturing, Springer

    Google Scholar 

  18. Song S, Chen H, Lin T, Wu D, Chen S (2016) Penetration state recognition based on the double-sound-sources characteristic of VPPAW and hidden Markov Model. J Mater Process Tech 234:33–44

    Article  Google Scholar 

  19. Zhang Z, Chen H, Xu Y, Zhong J, Lv N, Chen S (2015) Multisensor-based real-time quality monitoring by means of feature extraction, selection and modeling for Al alloy in arc welding. Mech Syst Signal Pr 60-61:151–165

    Article  Google Scholar 

  20. Chen C, Kovacevic R, Jandgric D (2003) Wavelet transform analysis of acoustic emission in monitoring friction stir welding of 6061 aluminum. Int J Mach Tool Manu 43(13):1383–1390

    Article  Google Scholar 

  21. Yao P, Zhou K, Zhu Q (2017) Quantitative evaluation method of arc sound spectrum based on sample entropy. Mech Syst Signal Pr 92:379–390

    Article  Google Scholar 

  22. Purtonen TKA, Salminen A (2014) Monitoring and adaptive control of laser processes. Physics Procedia 56:1218–1231

    Article  Google Scholar 

  23. Xiao X, Liu X, Cheng M, Song L (2020) Towards monitoring laser welding process via a coaxial pyrometer. J Mater Process Tech 277:116409

    Article  Google Scholar 

  24. Yang M, Bai R, Zheng H, Qi B (2020) Temperature monitoring and calibration in Ti-6Al-4V molten pool with pulsed arc welding. Sci Technol Weld Joi 25(5):369–376

    Article  Google Scholar 

  25. Kordi HA, Ghasempour-Mouziraji M, Hosseinzadeh M (2021) Metal transfer mapping for flux-cored arc welding process by using near-infrared filming. J Mater Eng Perform 30:3079–3095

    Article  Google Scholar 

  26. Jeong YC, Lee SJ, Park TW, Kim HK, Cho YT (2020) Real-time monitoring of immobilized oxide defects on bead surface in welding of high-strength galvanized steel. Int J Adv Manuf Tech 107(5-6):2409–2416

    Article  Google Scholar 

  27. Yang H, Wikle HC, Nagarajan S, Johnson M, Banerjee P, Chin BA (2008) 4 - Infrared sensors in welding. Real-Time Weld Process Monitoring: Woodhead Publishing, pp 74–103

  28. You D, Gao X, S K (2013) Multiple-optics sensing of high-brightness disk laser welding process. Ndt&e Int 60:32–39

    Article  Google Scholar 

  29. Chen H, Guo N, Xu K, Liu C, Wang G (2020) Investigating the advantages of ultrasonic-assisted welding technique applied in underwater wet welding by in-situ X-ray imaging method. Materials 13(6):1442

    Article  Google Scholar 

  30. Wang Z, Zhang C, Pan Z, Wang Z, Liu L, Qi X, Mao S, Pan J (2018) Image segmentation approaches for weld pool monitoring during robotic arc welding. Appl Sci-Basel 8(12):2445

    Article  Google Scholar 

  31. Kang S, Kang M, Jang YH, Kim C (2020) Droplet transfer and spatter generation in DC–AC pulse tandem gas metal arc welding. Sci Technol Weld Joi 25(7):589–599

    Article  Google Scholar 

  32. Diao C, Ding J, Williams S, Zhao Y (2017) A passive imaging system for geometry measurement for the plasma arc welding process. IEEE T Ind Electron 64(9):7201–7209

    Article  Google Scholar 

  33. Muhammad J, Altun H, Abo-Serie E (2017) Welding seam profiling techniques based on active vision sensing for intelligent robotic welding. The Int J Adv Manuf Tech 88(1-4):127–145

    Article  Google Scholar 

  34. Zhang Y, Zhang C, Tan L, Li S (2013) Coaxial monitoring of the fibre laser lap welding of Zn-coated steel sheets using an auxiliary illuminant. Opt Laser Technol 50:167–175

    Article  Google Scholar 

  35. Zhang Y, Liu T, Li B, Zhang Z (2019) Simultaneous monitoring of penetration status and joint tracking during laser keyhole welding. IEEE-Asme T Mech.

  36. Zhao H, Qi H (2016) Vision-based keyhole detection in laser full penetration welding process. J Laser Appl 28(2):022412

    Article  Google Scholar 

  37. You D, Gao X, Katayama S (2014) Visual-based spatter detection during high-power disk laser welding. Opt Laser Eng 54:1–7

    Article  Google Scholar 

  38. Chokkalingham S, Chandrasekhar N, Vasudevan M (2012) Predicting the depth of penetration and weld bead width from the infra red thermal image of the weld pool using artificial neural network modeling. J Intell Manuf 23(5):1995–2001

    Article  Google Scholar 

  39. Huang S-H, Huang Y-J, Hsieh C-H, Chen H-Z, Chui H-C (2018) Visual-assisted laser microwelding of carbon microfiber on metal plates. Opt Laser Technol 108:368–371

    Article  Google Scholar 

  40. Liu X, Jia C, Wu C (2017) Synchronous observation of topside weld pool, keyhole entrance and keyhole exit in keyhole plasma arc welding. 2017 IEEE 7th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). Hawaii, USA 344-348

  41. Lei Z, Shen J, Wang Q, Chen Y (2019) Real-time weld geometry prediction based on multi-information using neural network optimized by PCA and GA during thin-plate laser welding. J Manuf Process 43:207–217

    Article  Google Scholar 

  42. Ding S, You D, Cai F, Wu H, Gao X, Bai T (2020) Research on laser welding process and molding effect under energy deviation. Int J Adv Manuf Tech 108(5-6):1863–1874

    Article  Google Scholar 

  43. Zhang Y, Li F, Liang Z, Ying Y, Lin Q, Wei H (2018) Correlation analysis of penetration based on keyhole and plasma plume in laser welding. J Mater Process Tech 256:1–12

    Article  Google Scholar 

  44. Meng W, Yin X, Fang J, Guo L, Ma Q, Li Z (2019) Dynamic features of plasma plume and molten pool in laser lap welding based on image monitoring and processing techniques. Opt Laser Technol 109:168–177

    Article  Google Scholar 

  45. Cai W, Wang J, Cao L, Mi G, Shu L, Zhou Q, Jiang P (2019) Predicting the weld width from high-speed successive images of the weld zone using different machine learning algorithms during laser welding. MATH BIOSCI ENG 16(5):5595–5612

    Article  MathSciNet  Google Scholar 

  46. Chen Z, Gao X, Katayama S, Xiao Z, Chen X (2017) Elucidation of high-power disk laser welding phenomena by simultaneously observing both top and bottom of weldment. The Int J Adv Manuf Tech 88(1-4):1141–1150

    Article  Google Scholar 

  47. Gao X, Wang L, Chen Z, Zhang Y, You D (2020) Process stability analysis and weld formation evaluation during disk laser–mag hybrid welding. Opt Laser Eng 124:105835

    Article  Google Scholar 

  48. Lee SS, Kim TH, Hu SJ, Cai WW, Abell JA (2015) Analysis of weld formation in multilayer ultrasonic metal welding using high-speed images. J Manuf Sci E-T Asme 137(3):031016

    Article  Google Scholar 

  49. Chen C, Fan C, Cai X, Lin S, Yang C (2019) Analysis of droplet transfer, weld formation and microstructure in Al-Cu alloy bead welding joint with pulsed ultrasonic-GMAW method. J Mater Process Tech 271:144–151

    Article  Google Scholar 

  50. Chen C, Fan C, Lin S, Cai X, Zhou L, Ye S, Yang C (2018) Effect of ultrasonic pattern on weld appearance and droplet transfer in ultrasonic assisted MIG welding process. J Manuf Process 35:368–372

    Article  Google Scholar 

  51. Wang J, Sun Q, Zhang S, Wang C, Wu L, Feng J (2018) Characterization of the underwater welding arc bubble through a visual sensing method. J Mater Process Tech 251:95–108

    Article  Google Scholar 

  52. Jeong Y-C, Jung Y-G, Cho Y-T (2017) Simple monitoring of welding spatter for quantification and observation using a mobile phone. Adv Mech Eng 9(9):168781401772524

    Article  Google Scholar 

  53. Gao X, You D, Katayama S (2012) Infrared image recognition for seam tracking monitoring during fiber laser welding. Mechatronics 22(4):370–380

    Article  Google Scholar 

  54. Sreedhar U, Krishnamurthy C, Balasubramaniam K, Raghupathy V, Ravisankar S (2012) Automatic defect identification using thermal image analysis for online weld quality monitoring. J Mater Process Tech 212(7):1557–1566

    Article  Google Scholar 

  55. Liu Z, Wu CS, Gao J (2013) Vision-based observation of keyhole geometry in plasma arc welding. Int J Therm Sci 63:38–45

    Article  Google Scholar 

  56. Li C, Gao J, Cao Y, Yan X, Gui X (2019) Visual observation of fusion hole in thin plate TIG welding with a reserved gap. J Manuf Process 45:634–641

    Article  Google Scholar 

  57. Liu X, Wu C, Jia C, Zhang G (2017) Visual sensing of the weld pool geometry from the topside view in keyhole plasma arc welding. J Manuf Process 26:74–83

    Article  Google Scholar 

  58. Du R, Xu Y, Hou Z, Shu J, Chen S (2019) Strong noise image processing for vision-based seam tracking in robotic gas metal arc welding. The Int J Adv Manuf Tech 101(5-8):2135–2149

    Article  Google Scholar 

  59. Xiong J, Zou S (2019) Active vision sensing and feedback control of back penetration for thin sheet aluminum alloy in pulsed MIG suspension welding. Journal of Process Control 77:89–96

    Article  Google Scholar 

  60. Xu Y, Fang G, Chen S, Zou J, Ye Z (2014) Real-time image processing for vision-based weld seam tracking in robotic GMAW. The Int J Adv Manuf Tech 73(9-12):1413–1425

    Article  Google Scholar 

  61. Olsson R, Eriksson I, Powell J, Langtry A, Kaplan A (2011) Challenges to the interpretation of the electromagnetic feedback from laser welding. Opt Laser Eng 49(2):188–194

    Article  Google Scholar 

  62. Shen J, Zhao Z, Han J, Zhang Y, Bai L (2018) Dual-band welding speed monitoring method based on deep learning. Proceedings of the 2018 the 2nd International Conference on Video and Image Processing. Hong Kong, China 213-217

  63. Luo M, Shin YC (2015) Estimation of keyhole geometry and prediction of welding defects during laser welding based on a vision system and a radial basis function neural network. The Int J Adv Manuf Tech 81(1-4):263–276

    Article  Google Scholar 

  64. Kim C-H, Ahn D-C (2012) Coaxial monitoring of keyhole during Yb: YAG laser welding. Opt Laser Technol 44(6):1874–1880

    Article  Google Scholar 

  65. Luo M, Shin YC (2015) Vision-based weld pool boundary extraction and width measurement during keyhole fiber laser welding. Opt Laser Eng 64:59–70

    Article  Google Scholar 

  66. Meng Y, Gao M, Zeng X (2018) Quantitative analysis of synergic effects during laser-arc hybrid welding of AZ31 magnesium alloy. Opt Laser Eng 111:183–192

    Article  Google Scholar 

  67. Chen Z, Gao X (2014) Detection of weld pool width using infrared imaging during high-power fiber laser welding of type 304 austenitic stainless steel. The Int J Adv Manuf Tech 74(9-12):1247–1254

    Article  Google Scholar 

  68. Ding D (2017) Design of integrated neural network model for weld seam tracking and penetration monitoring. Cluster Comput 20(4):3345–3355

    Article  Google Scholar 

  69. Li M, Xiao R, Zou J, Wang L, Xu J, Zhang Q (2019) A multiple synchronous imaging method for strong illuminants induced during a hot working process. Laser Phys Lett 16(6):066003

    Article  Google Scholar 

  70. Kim T, Park Y (2014) Influence of welding parameters on weld bead in laser arc hybrid welding process using coaxial monitoring system and image processing. Mater. Res Innov 18(sup2):898–901

    Google Scholar 

  71. Liu S, Chen S, Wang Q, Li Y, Zhang H, Ding H (2017) Analysis of plasma characteristics and conductive mechanism of laser assisted pulsed arc welding. Opt Laser Eng 92:39–47

    Article  Google Scholar 

  72. Liu S, Zhang F, Dong S, Zhang H, Liu F (2018) Characteristics analysis of droplet transfer in laser-MAG hybrid welding process. Int J Heat Mass Tran 121:805–811

    Article  Google Scholar 

  73. Zou Y, Chen T (2018) Laser vision seam tracking system based on image processing and continuous convolution operator tracker. Opt Laser Eng 105:141–149

    Article  Google Scholar 

  74. Jun L, Xiao X (2018) Research on tube-plate welding robot based on binocular vision. 2018 International Symposium in Sensing and Instrumentation in IoT Era (ISSI). Shanghai, China 1-5

  75. Huang W, Kovacevic R (2012) Development of a real-time laser-based machine vision system to monitor and control welding processes. The Int J Adv Manuf Tech 63(1-4):235–248

    Article  Google Scholar 

  76. Lei T, Wang W, Rong Y, Xiong P, Huang Y (2020) Cross-lines laser aided machine vision in tube-to-tubesheet welding for welding height control. Opt Laser Technol 121:105796

    Article  Google Scholar 

  77. Wang ZZ (2014) Monitoring of GMAW weld pool from the reflected laser lines for real-time control. IEEE T Ind Inform 10(4):2073–2083

    Article  Google Scholar 

  78. Li C, Shi Y, Gu Y, Yuan P (2018) Monitoring weld pool oscillation using reflected laser pattern in gas tungsten arc welding. J Mater Process Tech 255:876–885

    Article  Google Scholar 

  79. Li C, Shi Y, Du L, Yufen G, Zhu M (2017) Real-time measurement of weld pool oscillation frequency in GTAW-P process. J Manuf Process 29:419–426

    Article  Google Scholar 

  80. Shao WJ, Huang Y, Zhang Y (2018) A novel weld seam detection method for space weld seam of narrow butt joint in laser welding. Opt Laser Technol 99:39–51

    Article  Google Scholar 

  81. Han Y, Fan J, Yang X (2020) A structured light vision sensor for on-line weld bead measurement and weld quality inspection. Int J Adv Manuf Tech 106(5-6):2065–2078

    Article  Google Scholar 

  82. Fidali M, Jamrozik W (2013) Diagnostic method of welding process based on fused infrared and vision images. Infrared Phys Techn 61:241–253

    Article  Google Scholar 

  83. Pinto-Lopera JE, Motta S, Mauricio J, Alfaro A, Crisostomo S (2016) Real-time measurement of width and height of weld beads in GMAW processes. Sensors-Basel 16(9):1500

    Article  Google Scholar 

  84. Gao J, Qin G, Yang J, He J, Zhang T, Wu C (2011) Image processing of weld pool and keyhole in Nd: YAG laser welding of stainless steel based on visual sensing. T Nonferr Metal Soc 21(2):423–428

    Article  Google Scholar 

  85. Gao X, Wen Q, Katayama S (2013) Analysis of high-power disk laser welding stability based on classification of plume and spatter characteristics. T Nonferr Metal Soc 23(12):3748–3757

    Article  Google Scholar 

  86. Wu Q, Lee J, Park M, Park C, Kim I, Wu Q, Lee J, Park M, Park C, Kim I (2014) A study on development of optimal noise filter algorithm for laser vision system in GMA welding. Procedia Engineering 97:819–827

    Article  Google Scholar 

  87. Jiang C, Zhang F, Wang Z (2017) Image processing of aluminum alloy weld pool for robotic VPPAW based on visual sensing. IEEE Access 5:21567–21573

    Article  Google Scholar 

  88. Gao X, Wang Y, Chen Z, Ma B, Zhang Y (2018) Analysis of welding process stability and weld quality by droplet transfer and explosion in MAG-laser hybrid welding process. J Manuf Process 32:522–529

    Article  Google Scholar 

  89. Hong Y, Chang B, Peng G, Yuan Z, Hou X, Xue B, Du D (2018) In-process monitoring of lack of fusion in ultra-thin sheets edge welding using machine vision. Sensors-Basel 18(8):2411

    Article  Google Scholar 

  90. Tenner F, Brock C, Klämpfl F, Schmidt M (2015) Analysis of the correlation between plasma plume and keyhole behavior in laser metal welding for the modeling of the keyhole geometry. Opt Laser Eng 64:32–41

    Article  Google Scholar 

  91. Gao X, Wang L, You D, Chen Z, Gao PP (2019) Synchronized monitoring of droplet transition and keyhole bottom in high power laser-MAG hybrid welding process. IEEE Sens J 19(9):3553–3563

    Article  Google Scholar 

  92. Haubold MW, Zäh MF (2019) Real-time spatter detection in laser welding with beam oscillation. Procedia CIRP 79:159–164

    Article  Google Scholar 

  93. Gao X, Zhang Y (2015) Monitoring of welding status by molten pool morphology during high-power disk laser welding. Optik 126(19):1797–1802

    Article  Google Scholar 

  94. Xia X, Jiang Z, Xu P (2019) A detection algorithm of spatter on welding plate surface based on machine vision. Optoelectronics Letters 15(1):52–56

    Article  Google Scholar 

  95. Pasinetti S, Sansoni G, Docchio F (2018) In-line monitoring of laser welding using a smart vision system. 2018 Workshop on Metrology for Industry 4.0 and IoT. Brescia, Italy 134-139

  96. You D, Gao X, Katayama S (2014) Monitoring of high-power laser welding using high-speed photographing and image processing. Mech Syst Signal Pr 49(1-2):39–52

    Article  Google Scholar 

  97. Zhu T, Shi Y, Cui S, Cui Y (2019) Recognition of weld penetration during K-TIG Welding based on acoustic and visual sensing. Sensing and Imaging 20(1):3

    Article  Google Scholar 

  98. Mikkelstrup A, Thomsen M, Stampe K, Endelt B, Boll J, Kristiansen E, Kristiansen M (2019) Quality inspection system for robotic laser welding of double-curved geometries. Procedia Manufacturing 36:50–57

    Article  Google Scholar 

  99. Nilsen M, Sikström F, Christiansson A-K, Ancona A (2017) Monitoring of varying joint gap width during laser beam welding by a dual vision and spectroscopic sensing system. Physics Procedia 89:100–107

    Article  Google Scholar 

  100. Zhang J, Yin H, Huang H, Yang R, Yang J, Li Y (2015) Feature extraction of molten pool for laser welding quality real-time inspection. Optica Applicata 45(4):523–533

    Google Scholar 

  101. Guo Y, Gao J, Cao Y, Li C (2019) Behavior of the fusion hole in tungsten inert gas thin-plate welding. IEEE Robot Autom Let 4(3):2801–2806

    Article  Google Scholar 

  102. Brock C, Tenner F, Klämpfl F, Hohenstein R, Schmidt M (2013) Detection of weld defects by high speed imaging of the vapor plume. Physics Procedia 41:539–543

    Article  Google Scholar 

  103. Ranjan R, Khan AR, Parikh C, Jain R, Mahto RP, Pal S, Pal SK, Chakravarty D (2016) Classification and identification of surface defects in friction stir welding: an image processing approach. J Manuf Process 22:237–253

    Article  Google Scholar 

  104. Zhang Y, Gao X (2014) Analysis of characteristics of molten pool using cast shadow during high-power disk laser welding. The Int J Adv Manuf Tech 70(9-12):1979–1988

    Article  Google Scholar 

  105. Jiao W, Wang Q, Cheng Y, Zhang Y (2020) End-to-end prediction of weld penetration: a deep learning and transfer learning based method. J Manuf Process 63:191–197

    Article  Google Scholar 

  106. Gunther J, Pilarski PM, Helfrich G, Shen H, Diepold K (2016) Intelligent laser welding through representation, prediction, and control learning: an architecture with deep neural networks and reinforcement learning. Mechatronics 34:1–11

    Article  Google Scholar 

  107. Zhang Y, Gao X, Katayama S (2015) Weld appearance prediction with BP neural network improved by genetic algorithm during disk laser welding. J Manuf Syst 34:53–59

    Article  Google Scholar 

  108. Jie S, Yan C, Fang L, Hua X (2017) Online detection method of weld penetration based on molten pool morphology and metallic vapor radiation for fiber laser welding. Int J Adv Manuf Tech 92(1-4):1–15

    Google Scholar 

  109. Liu G, Gao X, You D, Zhang N (2019) Prediction of high power laser welding status based on PCA and SVM classification of multiple sensors. J Intell Manuf 30:

  110. Wu D, Huang Y, Chen H, He Y, Chen S (2017) VPPAW penetration monitoring based on fusion of visual and acoustic signals using t-SNE and DBN model. Mater Design 123:1–14

    Article  Google Scholar 

  111. Zhang Z, Li B, Zhang W, Lu R, Wada S, Zhang Y (2020) Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks. J Manuf Syst 54:348–360

    Article  Google Scholar 

  112. Zhang Y, You D, Gao X, Wang C, Li Y, Gao PP (2019) Real-time monitoring of high-power disk laser welding statuses based on deep learning framework. J Intell Manuf 31(4):799–814

    Article  Google Scholar 

  113. Zhang Y, You D, Gao X, Katayama S (2019) Online monitoring of welding status based on a DBN model during laser welding. Engineering-Prc 5(4):671–678

    Google Scholar 

  114. Zhang Y, Gao X, You D, Ge W (2019) A low-cost welding status monitoring framework for high-power disk laser welding (December 2018). IEEE Access 7:17365–17376

    Article  Google Scholar 

  115. Wu D, Chen H, Huang Y, He Y, Hu M, Chen S (2017) Monitoring of weld joint penetration during variable polarity plasma arc welding based on the keyhole characteristics and PSO-ANFIS. J Mater Process Tech 239:113–124

    Article  Google Scholar 

  116. Zhang Z, Zhang Y, Luo F (2019) Convolutional neural network using Bayesian optimization for laser welding tailor rolled blanks penetration detection. 14th ASME International Manufacturing Science and Engineering Conference. Erie, PA

  117. Liang R, Yu R, Luo Y, Zhang Y (2019) Machine learning of weld joint penetration from weld pool surface using support vector regression. J Manuf Process 41:23–28

    Article  Google Scholar 

  118. Knaak C, Kolter G, Schulze F, Kröger M, Abels P (2019) Deep learning-based semantic segmentation for in-process monitoring in laser welding applications. Conference on Applications of Machine Learning. San Diego, CA 11139

  119. Gao X, Sun Y, Katayama S (2014) Neural network of plume and spatter for monitoring high-power disk laser welding. Int J Pr Eng Man-GT 1(4):293–298

    Google Scholar 

  120. Gao X, Zhang Y (2014) Prediction model of weld width during high-power disk laser welding of 304 austenitic stainless steel. Int J Precis Eng Man 15(3):399–405

    Article  Google Scholar 

  121. Li L, Xiao L, Liao H, Liu S, Ye B (2017) Welding quality monitoring of high frequency straight seam pipe based on image feature. J Mater Process Tech 246:285–290

    Article  Google Scholar 

  122. Haghshenas N, Moshayedi H (2019) Monitoring of resistance spot welding process. Exp Techniques 44(1):99–112

    Article  Google Scholar 

  123. Rubio-Solis A, Panoutsos G (2017) An ensemble data-driven fuzzy network for laser welding quality prediction. IEEE international conference on fuzzy systems (FUZZ-IEEE). Naples, Italy

  124. Cao L, Zhang L, Wu Y (2020) A data-driven model for weld bead monitoring during the laser welding assisted by magnetic field. Int J Adv Manuf Tech 107(1-2):475–487

    Article  Google Scholar 

  125. Liu T, Bao J, Wang J, Zhang Y (2018) A hybrid CNN–LSTM algorithm for online defect recognition of CO2 welding. Sensors-Basel 18(12):4369

    Article  Google Scholar 

  126. Zhang Z, Wen G, Chen S (2019) Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding. J Manuf Process 45:208–216

    Article  Google Scholar 

  127. Bacioiu D, Melton G, Papaelias M, Shaw R (2019) Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning. Ndt&e Int 107:102139

    Article  Google Scholar 

  128. Feng Y, Chen Z, Wang D, Chen J, Feng Z (2020) deepwelding: a deep learning enhanced approach to GTAW using multisource sensing images. IEEE T Ind Inform 16(1):465–474

    Article  Google Scholar 

  129. Shevchik SA, Lequang T, Meylan B, Farahani FV, Olbinado MP, Rack A, Masinelli G, Leinenbach C, Wasmer K (2020) Supervised deep learning for real-time quality monitoring of laser welding with X-ray radiographic guidance. Sci Rep-UK 10(1):1–12

    Google Scholar 

  130. Zhang Y, Zhang N, You D, Gao X, Katayama S (2019) High-power disk laser welding statuses monitoring based on analyses of multiple-sensor signals. J Manuf Process 41:221–230

    Article  Google Scholar 

  131. You D, Gao X, Katayama S (2015) WPD-PCA-based laser welding process monitoring and defects diagnosis by using FNN and SVM. IEEE T Ind Electron 62(1):628–636

    Article  Google Scholar 

  132. Chen J, Wang T, Gao X, Wei L (2018) Real-time monitoring of high-power disk laser welding based on support vector machine. Comput Ind 94:75–81

    Article  Google Scholar 

  133. Shevchik SA, Lequang T, Farahani FV, Faivre N, Meylan B, Zanoli S, Wasmer K (2019) Laser welding quality monitoring via graph support vector machine with data adaptive kernel. IEEE Access 7:93108–93122

    Article  Google Scholar 

  134. Sudhagar S, Sakthivel M, Ganeshkumar P (2019) Monitoring of friction stir welding based on vision system coupled with machine learning algorithm. Measurement 144:135–143

    Article  Google Scholar 

  135. Lei T, Rong Y, Xu J, Huang Y (2018) Experiment study and regression analysis of molten pool in laser welding. Opt Laser Technol 108:534–541

    Article  Google Scholar 

  136. Chen Z, Chen J, Feng Z (2018) Welding penetration prediction with passive vision system. J Manuf Process 36:224–230

    Article  Google Scholar 

  137. Huang J, Xue L, Huang J, Zou Y, Ma K (2019) Penetration estimation of GMA backing welding based on weld pool geometry parameters. Chin J Mech Eng-En 32(1):55–55

    Article  Google Scholar 

  138. Wu D, Chen H, Huang Y, He Y, Chen S (2016) Weld penetration identification for VPPAW based on keyhole features and extreme learning machine. 2016 IEEE workshop on advanced robotics and its social impacts (ARSO). IEEE. Shanghai, China 96-99

  139. Wu D, Chen H, Huang Y, Chen S (2019) Online monitoring and model-free adaptive control of weld penetration in VPPAW Based on Extreme Learning Machine. IEEE T Ind Inform 15(5):2732–2740

    Article  Google Scholar 

  140. Wu D, Chen J, Liu H, Zhang P, Yu Z, Chen H, Chen S (2019) Weld penetration in situ prediction from keyhole dynamic behavior under time-varying VPPAW pools via the OS-ELM model. The Int J Adv Manuf Tech 104:3929–3941

    Article  Google Scholar 

  141. You D, Gao X, Katayama S (2015) A novel stability quantification for disk laser welding by using frequency correlation coefficient between multiple-optics signals. IEEE-ASME T MECH 20(1):327–337

    Article  Google Scholar 

  142. Schweier M, Haubold MW, Zaeh MF (2016) Analysis of spatters in laser welding with beam oscillation: a machine vision approach. Cirp Journal of Manufacturing Science Technology 14:35–42

    Article  Google Scholar 

  143. Lu J, Xie H, Chen X, Han J, Bai L, Zhao Z (2020) Online welding quality diagnosis based on molten pool behavior prediction. Opt Laser Technol 126:106126

    Article  Google Scholar 

  144. Chen C, Lv N, Chen S (2020) Welding penetration monitoring for pulsed GTAW using visual sensor based on AAM and random forests. J Manuf Process 63:152–162

    Article  Google Scholar 

  145. Cai W, Wang J, Jiang P, Cao L, Qi Z (2020) Application of sensing techniques and artificial intelligence-based methods to laser welding real-time monitoring: a critical review of recent literature. J Manuf Syst 57:1–18

    Article  Google Scholar 

  146. Yang L, Liu Y, Peng J (2019) An automatic detection and identification method of welded joints based on deep neural network. IEEE Access 7:164952–164961

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the Guangzhou Municipal Special Fund Project for Scientific and Technological Innovation and Development under Grant 202002020068, 202002030147.

Author information

Authors and Affiliations

  1. Guangdong Provincial Welding Engineering Technology Research Center, Guangdong University of Technology, Guangzhou, 510006, China

    Xi’an Fan, Xiangdong Gao, Guiqian Liu, Nvjie Ma & Yanxi Zhang

Authors
  1. Xi’an Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiangdong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guiqian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nvjie Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanxi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed equally to this work.

Corresponding author

Correspondence to Xiangdong Gao.

Ethics declarations

Competing interests

The authors declare that they have no conflict of interest.

Consent to participate

Consent.

Consent to publish

Consent.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, X., Gao, X., Liu, G. et al. Research and prospect of welding monitoring technology based on machine vision. Int J Adv Manuf Technol 115, 3365–3391 (2021). https://doi.org/10.1007/s00170-021-07398-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-07398-4

Keywords

Search

Navigation