Skip to main content
SpringerLink
Log in

A robust weakly supervised learning of deep Conv-Nets for surface defect inspection

  • Review Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Automatic defect detection is a challenging task owing to the complex textured background with non-uniform intensity distribution, weak differences between defects and background, diversity of defect types, and high cost of annotated samples. In order to solve these challenges, this paper proposes a novel end-to-end defect classification and segmentation framework based on weakly supervised learning of a convolutional neural network (CNN) with attention architecture. Firstly, a novel end-to-end CNN architecture integrating the robust classifier and spatial attention module is proposed to enhance defect feature representation ability, which significantly improves the classification accuracy. Secondly, a new spatial attention class activation map (SA-CAM) is proposed to improve segmentation adaptability by generating more accurate heatmap. Moreover, for different surface texture, SA-CAM can significantly suppress the background’s inference and highlight defect area. Finally, the proposed weakly supervised learning framework is trained using only global image labels and devoted to two main visual recognition tasks: defect samples classification and area segmentation. At the same time, it is robust to complex backgrounds. Results of the experiments verify the generalization of the proposed method on three distinct datasets with different kinds of textures and backgrounds. In the classification tasks, the proposed method improves accuracy by 0.66–25.50%. In the segmentation tasks, the proposed method improves accuracy by 5.49–7.07%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Xie X (2008) Review of recent advances in surface defect detection using texture analysis techniques. ELCVIA: Electron Lett Comput Vis Image Anal 7(3):1

    Article  Google Scholar 

  2. Luo Q, Sun Y, Li P, Simpson O, Tian L, He Y (2018) Generalized completed local binary patterns for time-efficient steel surface defect classification. IEEE Trans Instrum Meas 68(3):667

    Article  Google Scholar 

  3. Binyi S et al (2019) Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor. IEEE Trans Instrum Meas 68(12):4675–4688

    Article  Google Scholar 

  4. Yapi D, Allili MS, Baaziz N (2017) Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Trans Autom Sci Eng 15(3):1014

    Article  Google Scholar 

  5. Wang H, Qi H, Wang XF (2013) A new Gabor based approach for wood recognition. Neurocomputing 116:192

    Article  Google Scholar 

  6. Zhang Z, Zou Y, Gan C (2018) Textual sentiment analysis via three different attention convolutional neural networks and cross-modality consistent regression. Neurocomputing 275:1407

    Article  Google Scholar 

  7. Xie L, Huang R, Gu N, Cao Z (2014) A novel defect detection and identification method in optical inspection. Neural Comput Appl 24(7):1953

    Article  Google Scholar 

  8. Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE (2017) Survey of deep neural network architectures and their applications. Neurocomputing 234:11

    Article  Google Scholar 

  9. Mirjalili SM, Mirjalili SZ (2017) Single-objective optimization framework for designing photonic crystal filters. Neural Comput Appl 28(6):1463

    Article  Google Scholar 

  10. Jung S, Tsai Y, Chiu W, Hu J, Sun C (2018) Defect detection on randomly textured surfaces by convolutional neural networks. In: 2018 IEEE/ASME international conference on advanced intelligent mechatronics (AIM) (IEEE, 2018), pp 1456–1461

  11. Chen H, Pang Y, Hu Q, Liu K (2020) Solar cell surface defect inspection based on multispectral convolutional neural network. J Intell Manuf 31:453–468

    Article  Google Scholar 

  12. Zhou S, Chen Y, Zhang D, Xie J, Zhou Y (2017) Classification of surface defects on steel sheet using convolutional neural networks. Mater Technol 51(1):123

    Google Scholar 

  13. Tang Y (2013) Deep learning using linear support vector machines. arXiv:1306.0239

  14. Merentitis A, Debes C (2015) Automatic fusion and classification using random forests and features extracted with deep learning. In: 2015 IEEE international geoscience and remote sensing symposium (IGARSS) (IEEE, 2015), pp 2943–2946

  15. Zhang H, Zhang L, Li P, Gu D (2018) Yarn-dyed fabric defect detection with yolov2 based on deep convolution neural networks. In: 2018 IEEE 7th data driven control and learning systems conference (DDCLS) (IEEE, 2018), pp 170–174

  16. Singh J, Shekhar S (2018) Road damage detection and classification in smartphone captured images using mask r-cnn. arXiv:1811.04535

  17. Yuille AL, Liu C (2018) Deep nets: What have they ever done for vision? arXiv:1805.04025

  18. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

  19. Ren R, Hung T, Tan KC (2017) Generic deep-learning-based approach for automated surface inspection. IEEE Trans Cybern 48(3):929

    Article  Google Scholar 

  20. Lin H, Li B, Wang X, Shu Y, Niu S (2019) Automated defect inspection of led chip using deep convolutional neural network. J Intell Manuf 30(6):2525

    Article  Google Scholar 

  21. Li W, Leonardis A, Fritz M (2017) Visual stability prediction and its application to manipulation. In: 2017 AAAI Spring symposium series

  22. Jaderberg M, Simonyan K, Zisserman A et al (2015) Spatial transformer networks. In: Advances in neural information processing systems, pp 2017–2025

  23. Ji Y, Zhang H, Wu QMJ (2018) Salient object detection via multi-scale attention CNN. Neurocomputing 322:130–140

    Article  Google Scholar 

  24. Breiman L (2001) Random forests. Mach Learn 45(1):5

    Article  MATH  Google Scholar 

  25. Kairanbay M, See J, Wong LK, Hii YL (2017) Filling the gaps: reducing the complexity of networks for multi-attribute image aesthetic prediction. In: 2017 IEEE international conference on image processing (ICIP) (IEEE, 2017), pp 3051–3055

  26. Otsu N (1979) A threshold selection method from gray-level histogram. IEEE Trans Syst Man Cybern 9(1):62

    Article  Google Scholar 

  27. Silven O, Niskanen M, Kauppinen H (2003) Wood inspection with non-supervised clustering. Mach Vis Appl 13(5–6):275

    Article  Google Scholar 

  28. Wang T, Chen Y, Qiao M, Snoussi H (2018) A fast and robust convolutional neural network-based defect detection model in product quality control. Int J Adv Manuf Technol 94(9–12):3465

    Article  Google Scholar 

  29. Zhang H et al (2018) Tree2Vector: learning a vectorial representation for tree-structured data. IEEE Trans Neural Netw Learn Syst 99:1–15

    MathSciNet  Google Scholar 

  30. Zhai W, Zhu J, Cao Y, Wang Z (2018) A generative adversarial network-based framework for unsupervised visual surface inspection. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP) (IEEE, 2018), pp 1283–1287

  31. Zhang J, Sclaroff S (2015) Exploiting surroundedness for saliency detection: a Boolean map approach. IEEE Trans Pattern Anal Mach Intell 38(5):889

    Article  Google Scholar 

  32. Donoser M, Bischof H (2008) Using covariance matrices for unsupervised texture segmentation. In: 2008 19th international conference on pattern recognition (IEEE, 2008), pp 1–4

Download references

Acknowledgements

This work is supported in part by National Natural Science Foundation (NNSF) of China under Grant 61873315, Natural Science Foundation of Hebei Province under Grant F2018202078, Science and Technology Program of Hebei Province under Grant 17211804D, Hebei Province Outstanding Youth Science Foundation F2017202062 and Young Talents Project in Hebei Province under Grant 210003.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, 300000, China

    Haiyong Chen, Qidi Hu, Baoshuo Zhai, He Chen & Kun Liu

  2. State Key Laboratory for Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin, 300000, China

    Haiyong Chen

Authors
  1. Haiyong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qidi Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baoshuo Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. He Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haiyong Chen.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service, and company that could be construed as influencing the position presented in or the review of the manuscript entitled.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Hu, Q., Zhai, B. et al. A robust weakly supervised learning of deep Conv-Nets for surface defect inspection. Neural Comput & Applic 32, 11229–11244 (2020). https://doi.org/10.1007/s00521-020-04819-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-04819-5

Keywords

Search

Navigation