Elsevier

Expert Systems with Applications

Volume 128, 15 August 2019, Pages 84-95
Expert Systems with Applications

An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification

https://doi.org/10.1016/j.eswa.2019.01.048Get rights and content

Highlights

  • A novel interpretable hierarchical deep learning model for lung cancer diagnosis.

  • Network design incorporates semantic features that are intuitive to radiologists.

  • A single joint network predicts interpretable features and malignancy.

  • Architecture maintains prediction accuracy while improving model interpretability.

Abstract

While deep learning methods have demonstrated performance comparable to human readers in tasks such as computer-aided diagnosis, these models are difficult to interpret, do not incorporate prior domain knowledge, and are often considered as a “black-box.” The lack of model interpretability hinders them from being fully understood by end users such as radiologists. In this paper, we present a novel interpretable deep hierarchical semantic convolutional neural network (HSCNN) to predict whether a given pulmonary nodule observed on a computed tomography (CT) scan is malignant. Our network provides two levels of output: 1) low-level semantic features; and 2) a high-level prediction of nodule malignancy. The low-level outputs reflect diagnostic features often reported by radiologists and serve to explain how the model interprets the images in an expert-interpretable manner. The information from these low-level outputs, along with the representations learned by the convolutional layers, are then combined and used to infer the high-level output. This unified architecture is trained by optimizing a global loss function including both low- and high-level tasks, thereby learning all the parameters within a joint framework. Our experimental results using the Lung Image Database Consortium (LIDC) show that the proposed method not only produces interpretable lung cancer predictions but also achieves better results compared to using a 3D CNN alone.

Section snippets

Introduction and background

Lung cancer is the leading cause of cancer mortality worldwide (Torre, Siegel, & Jemal, 2016). Computed tomography (CT) imaging is increasingly being used to detect and characterize pulmonary nodules with the purpose of diagnosing lung cancer earlier. The National Lung Screening Trial (NLST) (Team et al., 2011) in the United States demonstrated a 20% lung cancer mortality reduction in high-risk subjects who underwent screening using low-dose CT relative to plain chest radiography. Based on the

Lung image database consortium dataset

The Lung Image Database Consortium image collection (LIDC-IDRI) (Armato et al., 2011) is a publicly available dataset, which we used to train and test our proposed methods. LIDC-IDRI contains both screening and diagnostic CT scans collected from 7 academic centers and 8 medical imaging companies. Inclusion criteria for CT scans were: 1) having a collimation and reconstruction interval no greater than 3 mm; and 2) each scan approximately containing no more than 6 lung nodules with the longest

Experimental results

This section first describes how we trained the models. We compare our model to a traditional 3D CNN model and other state-of-the-art methods. We also evaluate the accuracy of semantic feature predictions, providing illustrations of correct and incorrect predictions (Table 4).

Discussion

We present the HSCNN model that incorporates domain knowledge into the model architecture design, predicting semantic nodule characteristics along with the primary task of nodule malignancy diagnosis. Five semantic features were considered: calcification, margin, subtlety, texture, and sphericity. Our results in Section 3.3 suggest that the HSCNN model is capable of providing accurate predictions of semantic descriptors while simultaneously classifying nodule malignancy. The semantic labels are

Conclusion

In this paper, we have developed a novel radiologist-interpretable HSCNN model for predicting lung cancer in CT-detected indeterminate nodules. This model is able to simultaneously predict nodule malignancy while classifying five nodule semantic characteristics, including calcification, margin, subtlety, texture, and sphericity of nodules. These diagnostic semantic features predictions are intermediate outputs associated with the final malignancy prediction and are useful to explain the

Author contributions

All authors contributed to the development of the project. SS developed the methodology, conducted the experiments and wrote the manuscript. SXH contributed to the experiments. AAB and DRA provided valuable advice and domain input. WH provided oversight over the project and contributed to its design. All authors reviewed the manuscript.

Conflict of interest

None declared.

CRediT authorship contribution statement

Shiwen Shen: Conceptualization, Data curation, Investigation, Methodology, Writing - original draft, Writing - review & editing. Simon X Han: Investigation, Methodology, Writing - review & editing. Denise R Aberle: Funding acquisition, Writing - review & editing. Alex A Bui: Methodology, Writing - review & editing. William Hsu: Conceptualization, Methodology, Project administration, Writing - original draft, Writing - review & editing.

Acknowledgements

The authors acknowledge the National Cancer Institute and the Foundation for the National Institutes of Health, and their critical role in the creation of the free publicly available LIDC/IDRI Database used in this study. Research reported in this publication was partly supported by the National Cancer Institute of the National Institutes of Health under award number R01 CA210360, the Center for Domain-Specific Computing (CDSC) funded by the National Science Foundation under grant no. 1436827

References (44)

  • M. Abadi et al.

    Tensorflow: A system for large-scale machine learning

    Osdi

    (2016)
  • S.G. Armato et al.

    Automated lung nodule classification following automated nodule detection on ct: A serial approach

    Medical Physics

    (2003)
  • S.G. Armato et al.

    The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans

    Medical Physics

    (2011)
  • J. Bergstra et al.

    Random search for hyper-parameter optimization

    Journal of Machine Learning Research

    (2012)
  • Chollet, F. et al. (2015)....
  • K. Clark et al.

    The cancer imaging archive (TCIA): Maintaining and operating a public information repository

    Journal of Digital Imaging

    (2013)
  • N. Duggan et al.

    A technique for lung nodule candidate detection in ct using global minimization methods

    International workshop on energy minimization methods in computer vision and pattern recognition

    (2015)
  • J.J. Erasmus et al.

    Solitary pulmonary nodules: Part i. morphologic evaluation for differentiation of benign and malignant lesions

    Radiographics

    (2000)
  • A. Farag et al.

    Evaluation of geometric feature descriptors for detection and classification of lung nodules in low dose ct scans of the chest

    Biomedical imaging: From nano to macro, 2011 IEEE international symposium on

    (2011)
  • M. Firmino et al.

    Computer-aided detection (CADe) and diagnosis (CADx) system for lung cancer with likelihood of malignancy

    Biomedical Engineering Online

    (2016)
  • X. Glorot et al.

    Understanding the difficulty of training deep feedforward neural networks

    Proceedings of the thirteenth international conference on artificial intelligence and statistics

    (2010)
  • K. ten Haaf et al.

    Risk prediction models for selection of lung cancer screening candidates: A retrospective validation study

    PLoS Medicine

    (2017)
  • Cited by (196)

    • Atrous convolution aided integrated framework for lung nodule segmentation and classification

      2023, Biomedical Signal Processing and Control
      Citation Excerpt :

      An accuracy of 92.53 % is achieved by the proposed model. In [18], a novel interpretable deep learning framework is introduced for nodule characterization. The work combines low-level diagnostic features with the features obtained from convolutional layers.

    View all citing articles on Scopus
    View full text