Skip to main content
SpringerLink
Log in

Pulmonary Nodule Classification Using a Multiview Residual Selective Kernel Network

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

Lung cancer is one of the leading causes of death worldwide and early detection is crucial to reduce the mortality. A reliable computer-aided diagnosis (CAD) system can help facilitate early detection of malignant nodules. Although existing methods provide adequate classification accuracy, there is still room for further improvement. This study is dedicated to investigating a new CAD scheme for predicting the malignant likelihood of lung nodules in computed tomography (CT) images in light of a deep learning strategy. Conceived from the residual learning and selective kernel, we investigated an efficient residual selective kernel (RSK) block to handle the diversity of lung nodules with various shapes and obscure structures. Founded on this RSK block, we established a multiview RSK network (MRSKNet), to which three anatomical planes in the axial, coronal, and sagittal directions were fed. To reinforce the classification efficiency, seven handcrafted texture features with a filter-like computation strategy were explored, among which the homogeneity (HOM) feature maps are combined with the corresponding intensity CT images for concatenation input, leading to an improved network architecture. Evaluated on the public benchmark Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) challenge database with ten-fold cross validation of binary classification, our experimental results indicated high area under receiver operating characteristic (AUC) and accuracy scores. A better compromise between recall and specificity was struck using the suggested concatenation strategy comparing to many state-of-the-art approaches. The proposed pulmonary nodule classification framework exhibited great efficacy and achieved a higher AUC of 0.9711. The association of handcrafted texture features with deep learning models is promising in advancing the classification performance. The developed pulmonary nodule CAD network architecture is of potential in facilitating the diagnosis of lung cancer for further image processing applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The inclusion of datasets is future work, which will be collected from any possible public domain.

References

  1. Ferlay J, et al.: Cancer statistics for the year 2020: An overview. International Journal of Cancer 149:778-789, 2021

    Article  CAS  Google Scholar 

  2. Siegel RL, Miller KD, Fuchs HE, Jemal A: Cancer Statistics, 2021. CA: A Cancer Journal for Clinicians 71:7–33, 2021

  3. Lu S, et al.: Iterative reconstruction of low-dose CT based on differential sparse. Biomedical Signal Processing and Control 79:104204, 2023

    Article  Google Scholar 

  4. Jennifer S, Sharmila S: A Neutrosophic Set Approach on Chest X-rays for Automatic Lung Infection Detection. Information Technology and Control 52:37-52, 2023

    Article  Google Scholar 

  5. Poap D, Wozniak M, Damaševičius R, Wei W: Chest radiographs segmentation by the use of nature-inspired algorithm for lung disease detection. Proc. 2018 IEEE Symposium Series on Computational Intelligence (SSCI): City, 18–21 Nov. 2018

  6. Jaszcz A, Połap D, Damaševičius R: Lung X-Ray Image Segmentation Using Heuristic Red Fox Optimization Algorithm. Scientific Programming 2022:4494139, 2022

    Article  Google Scholar 

  7. Ma H, et al.: Automatic pulmonary ground-glass opacity nodules detection and classification based on 3D neural network. Medical Physics 49:2555-2569, 2022

    Article  ADS  PubMed  Google Scholar 

  8. Thakur SK, Singh DP, Choudhary J: Lung cancer identification: a review on detection and classification. Cancer and Metastasis Reviews 39:989-998, 2020

    Article  PubMed  Google Scholar 

  9. Zhang G, Yang Z, Gong L, Jiang S, Wang L, Zhang H: Classification of lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations. La radiologia medica 125:374-383, 2020

    Article  PubMed  Google Scholar 

  10. Saba T: Automated lung nodule detection and classification based on multiple classifiers voting. Microscopy Research and Technique 82:1601-1609, 2019

    Article  PubMed  Google Scholar 

  11. Muzammil M, Ali I, Haq IU, Khaliq AA, Abdullah S: Pulmonary Nodule Classification Using Feature and Ensemble Learning-Based Fusion Techniques. IEEE Access 9:113415-113427, 2021

    Article  Google Scholar 

  12. Farag AA, Ali A, Elshazly S, Farag AA: Feature fusion for lung nodule classification. International Journal of Computer Assisted Radiology and Surgery 12:1809-1818, 2017

    Article  PubMed  Google Scholar 

  13. Chen S, et al.: Automatic Scoring of Multiple Semantic Attributes With Multi-Task Feature Leverage: A Study on Pulmonary Nodules in CT Images. IEEE Transactions on Medical Imaging 36:802-814, 2017

    Article  Google Scholar 

  14. Wei G, Cao H, Ma H, Qi S, Qian W, Ma Z: Content-based image retrieval for Lung Nodule Classification Using Texture Features and Learned Distance Metric. Journal of Medical Systems 42:13, 2017

    Article  PubMed  Google Scholar 

  15. Zhang F, et al.: Lung Nodule Classification With Multilevel Patch-Based Context Analysis. IEEE Transactions on Biomedical Engineering 61:1155-1166, 2014

    Article  PubMed  Google Scholar 

  16. Mao K, Deng Z: Lung Nodule Image Classification Based on Local Difference Pattern and Combined Classifier. Computational and Mathematical Methods in Medicine 2016:1091279, 2016

    Article  PubMed  PubMed Central  Google Scholar 

  17. Cicero M, et al.: Training and Validating a Deep Convolutional Neural Network for Computer-Aided Detection and Classification of Abnormalities on Frontal Chest Radiographs. Investigative Radiology 52:281-287, 2017

    Article  PubMed  Google Scholar 

  18. Khan MA, et al.: VGG19 Network Assisted Joint Segmentation and Classification of Lung Nodules in CT Images. Diagnostics 11:2208, 2021

    Article  PubMed  PubMed Central  Google Scholar 

  19. He K, Zhang X, Ren S, Sun J: Deep residual learning for image recognition. Proc. Proceedings of the IEEE conference on computer vision and pattern recognition: City

  20. Li X, Wang W, Hu X, Yang J: Selective kernel networks. Proc. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition: City

  21. Liu M, et al.: Three-Dimensional Modeling of Heart Soft Tissue Motion. Applied Sciences 13:2493, 2023

    Article  CAS  Google Scholar 

  22. Liu X, Yang L, Chen J, Yu S, Li K: Region-to-boundary deep learning model with multi-scale feature fusion for medical image segmentation. Biomedical Signal Processing and Control 71:103165, 2022

    Article  Google Scholar 

  23. Uthoff J, et al.: Machine learning approach for distinguishing malignant and benign lung nodules utilizing standardized perinodular parenchymal features from CT. Medical Physics 46:3207-3216, 2019

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  24. Zheng B, Yang D, Zhu Y, Liu Y, Hu J, Bai C: 3D gray density coding feature for benign-malignant pulmonary nodule classification on chest CT. Medical Physics 48:7826-7836, 2021

    Article  ADS  PubMed  Google Scholar 

  25. Dhara AK, Mukhopadhyay S, Dutta A, Garg M, Khandelwal N: A Combination of Shape and Texture Features for Classification of Pulmonary Nodules in Lung CT Images. Journal of Digital Imaging 29:466-475, 2016

    Article  PubMed  PubMed Central  Google Scholar 

  26. Madero Orozco H, Vergara Villegas OO, Cruz Sánchez VG, Ochoa Domínguez HdJ, Nandayapa Alfaro MdJ: Automated system for lung nodules classification based on wavelet feature descriptor and support vector machine. BioMedical Engineering OnLine 14:9, 2015

    Article  PubMed  PubMed Central  Google Scholar 

  27. de Sousa Costa RW, da Silva GLF, de Carvalho Filho AO, Silva AC, de Paiva AC, Gattass M: Classification of malignant and benign lung nodules using taxonomic diversity index and phylogenetic distance. Med Biol Eng Comput 56:2125-2136, 2018

    Article  PubMed  Google Scholar 

  28. Firmino M, Angelo G, Morais H, Dantas MR, Valentim R: Computer-aided detection (CADe) and diagnosis (CADx) system for lung cancer with likelihood of malignancy. BioMedical Engineering OnLine 15:2, 2016

    Article  PubMed  PubMed Central  Google Scholar 

  29. de Carvalho Filho AO, Silva AC, Cardoso de Paiva A, Nunes RA, Gattass M: Computer-Aided Diagnosis of Lung Nodules in Computed Tomography by Using Phylogenetic Diversity, Genetic Algorithm, and SVM. Journal of Digital Imaging 30:812-822, 2017

    Article  PubMed  PubMed Central  Google Scholar 

  30. Sasidhar B, Geetha G, Khodanpur B, Babu DR: Automatic classification of lung nodules into benign or malignant using SVM classifier. Proc. Proceedings of the 5th International Conference on Frontiers in Intelligent Computing: Theory and Applications: City

  31. Li X-X, Li B, Tian L-F, Zhang L: Automatic benign and malignant classification of pulmonary nodules in thoracic computed tomography based on RF algorithm. IET Image Processing 12:1253-1264, 2018

    Article  Google Scholar 

  32. Wu W, Hu H, Gong J, Li X, Huang G, Nie S: Malignant-benign classification of pulmonary nodules based on random forest aided by clustering analysis. Physics in Medicine & Biology 64:035017, 2019

    Article  ADS  Google Scholar 

  33. Rodrigues MB, et al.: Health of things algorithms for malignancy level classification of lung nodules6:18592–18601, 2018

  34. Lee MC, et al.: Computer-aided diagnosis of pulmonary nodules using a two-step approach for feature selection and classifier ensemble construction. Artificial Intelligence in Medicine 50:43-53, 2010

    Article  PubMed  Google Scholar 

  35. Farahani FV, Ahmadi A, Zarandi MHF: Hybrid intelligent approach for diagnosis of the lung nodule from CT images using spatial kernelized fuzzy c-means and ensemble learning. Mathematics and Computers in Simulation 149:48-68, 2018

    Article  MathSciNet  Google Scholar 

  36. Suganyadevi S, Seethalakshmi V, Balasamy K: A review on deep learning in medical image analysis. International Journal of Multimedia Information Retrieval 11:19-38, 2022

    Article  CAS  PubMed  Google Scholar 

  37. Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks, 2012

  38. Simonyan K, Zisserman A: Very deep convolutional networks for large-scale image recognition. arXiv:14091556, 2014

  39. Szegedy C, et al.: Going deeper with convolutions. Proc. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): City

  40. Li R, Xiao C, Huang Y, Hassan H, Huang B: Deep Learning Applications in Computed Tomography Images for Pulmonary Nodule Detection and Diagnosis: A Review. Diagnostics 12:298, 2022

    Article  PubMed  PubMed Central  Google Scholar 

  41. Murugesan M, Kaliannan K, Balraj S, Singaram K, Kaliannan T, Albert JR: A Hybrid deep learning model for effective segmentation and classification of lung nodules from CT images. Journal of Intelligent & Fuzzy Systems 42:2667-2679, 2022

    Article  Google Scholar 

  42. Lu S, et al.: Soft Tissue Feature Tracking Based on Deep Matching Network. Computer Modeling in Engineering \& Sciences 136:363--379, 2023

  43. Dai Y, Yan S, Zheng B, Song C: Incorporating automatically learned pulmonary nodule attributes into a convolutional neural network to improve accuracy of benign-malignant nodule classification. Physics in Medicine & Biology 63:245004, 2018

    Article  ADS  Google Scholar 

  44. Ren Y, et al.: A manifold learning regularization approach to enhance 3D CT image-based lung nodule classification. International Journal of Computer Assisted Radiology and Surgery 15:287-295, 2020

    Article  PubMed  Google Scholar 

  45. Xie S, Girshick R, Dollár P, Tu Z, He K: Aggregated Residual Transformations for Deep Neural Networks. Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): City, 21–26 July 2017

  46. Liu H, et al.: Multi-model Ensemble Learning Architecture Based on 3D CNN for Lung Nodule Malignancy Suspiciousness Classification. Journal of Digital Imaging 33:1242-1256, 2020

    Article  PubMed  PubMed Central  Google Scholar 

  47. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA: Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Proc. AAAI Conference on Artificial Intelligence: City

  48. Zuo W, Zhou F, He Y, Li X: Automatic classification of lung nodule candidates based on a novel 3D convolution network and knowledge transferred from a 2D network. Medical Physics 46:5499-5513, 2019

    Article  ADS  PubMed  Google Scholar 

  49. Lyu J, Bi X, Ling SH: Multi-Level Cross Residual Network for Lung Nodule Classification. Sensors 20:2837, 2020

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  50. An Y, Hu T, Wang J, Lyu J, Banerjee S, Ling SH: Lung Nodule Classification using A Novel Two-stage Convolutional Neural Networks Structure’. Proc. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC): City

  51. Su H, Maji S, Kalogerakis E, Learned-Miller E: Multi-view convolutional neural networks for 3d shape recognition. Proc. Proceedings of the IEEE international conference on computer vision: City

  52. Nibali A, He Z, Wollersheim D: Pulmonary nodule classification with deep residual networks. International Journal of Computer Assisted Radiology and Surgery 12:1799-1808, 2017

    Article  PubMed  Google Scholar 

  53. Sahu P, Yu D, Dasari M, Hou F, Qin H: A Lightweight Multi-Section CNN for Lung Nodule Classification and Malignancy Estimation. IEEE Journal of Biomedical and Health Informatics 23:960-968, 2019

    Article  PubMed  Google Scholar 

  54. Al-Shabi M, Lee HK, Tan M: Gated-Dilated Networks for Lung Nodule Classification in CT Scans. IEEE Access 7:178827-178838, 2019

    Article  Google Scholar 

  55. Al-Shabi M, Lan BL, Chan WY, Ng K-H, Tan M: Lung nodule classification using deep Local–Global networks. International Journal of Computer Assisted Radiology and Surgery 14:1815-1819, 2019

    Article  PubMed  Google Scholar 

  56. Xie Y, et al.: Knowledge-based Collaborative Deep Learning for Benign-Malignant Lung Nodule Classification on Chest CT. IEEE Transactions on Medical Imaging 38:991-1004, 2019

    Article  PubMed  Google Scholar 

  57. Xie Y, Zhang J, Xia Y: Semi-supervised adversarial model for benign–malignant lung nodule classification on chest CT. Medical Image Analysis 57:237-248, 2019

    Article  PubMed  Google Scholar 

  58. Wang C, Elazab A, Wu J, Hu Q: Lung nodule classification using deep feature fusion in chest radiography. Computerized Medical Imaging and Graphics 57:10-18, 2017

    Article  PubMed  Google Scholar 

  59. Zhu W, Liu C, Fan W, Xie X: Deeplung: Deep 3d dual path nets for automated pulmonary nodule detection and classification. Proc. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV): City

  60. Friedman JH: Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics 29:1189-1232, 2001

    Article  MathSciNet  Google Scholar 

  61. Zhang G, Yang Z, Gong L, Jiang S, Wang L: Classification of benign and malignant lung nodules from CT images based on hybrid features. Physics in Medicine & Biology 64:125011, 2019

    Article  ADS  Google Scholar 

  62. Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J: Dual path networks, Long Beach, California, USA: Curran Associates Inc., 2017

    Google Scholar 

  63. Huang G, Liu Z, Maaten LVD, Weinberger KQ: Densely Connected Convolutional Networks. Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR): City, 21–26 July 2017

  64. Chang H, Hsieh C: Brain segmentation in MR images using a texture-based classifier associated with mathematical morphology. Proc. 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC): City

  65. Haralick RM, Shanmugam K, Dinstein I: Textural Features for Image Classification. IEEE Transactions on Systems, Man, and Cybernetics SMC-3:610–621, 1973

  66. Galloway MM: Texture analysis using gray level run lengths. Computer Graphics and Image Processing 4:172-179, 1975

    Article  Google Scholar 

  67. Tamura H, Mori S, Yamawaki T: Textural Features Corresponding to Visual Perception. IEEE Transactions on Systems, Man, and Cybernetics 8:460-473, 1978

    Article  Google Scholar 

  68. Armato III SG, et al.: The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans. Medical Physics 38:915-931, 2011

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  69. Clark K, et al.: The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. Journal of Digital Imaging 26:1045-1057, 2013

    Article  PubMed  PubMed Central  Google Scholar 

  70. Reeves AP, Biancardi AM: The Lung Image Database Consortium (LIDC) Nodule Size Report. Release: 2011–10–27–2 At: /lidc/, 2011

Download references

Funding

This work was supported by the Ministry of Science and Technology of Taiwan under Grant No. MOST 108-2221-E-002-080-MY3.

Author information

Authors and Affiliations

  1. Computational Biomedical Engineering Laboratory (CBEL), Department of Engineering Science and Ocean Engineering, National Taiwan University, 1 Sec. 4 Roosevelt Road, Daan, Taipei, 10617, Taiwan

    Herng-Hua Chang & Cheng-Zhe Wu

  2. Department of Radiation Oncology, Keck Medical School, University of Southern California, Los Angeles, CA, USA

    Audrey Haihong Gallogly

Authors
  1. Herng-Hua Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cheng-Zhe Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Audrey Haihong Gallogly
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Herng-Hua Chang.

Ethics declarations

Ethics Approval

This paper does not contain any studies with human participants or animals performed by any of the authors.

Consent to Participate

This paper does not contain patient data.

Competing Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, HH., Wu, CZ. & Gallogly, A.H. Pulmonary Nodule Classification Using a Multiview Residual Selective Kernel Network. J Digit Imaging. Inform. med. 37, 347–362 (2024). https://doi.org/10.1007/s10278-023-00928-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-023-00928-4

Keywords

Search

Navigation