From 3D to 2D: Transferring knowledge for rib segmentation in chest X-rays☆
Introduction
Deep Neural Networks (DNNs) have achieved state-of-the-art results in both computer vision and biomedical image analysis [1]. DNNs are powerful overcomplete statistical models that can learn to extract features from unstructured data such as image, audio, video and text. These models can be understood as ensembles of perceptrons organized in layers with increasing semantic capabilities, being able to extract and select features and to perform inference conjointly. In general, deeper models are able to encode information with higher semantics, while shallower models can only optimize low-level semantic information. Convolutional Neural Networks (CNNs) [2] and their variants [3], [4] are the most popular architectures used for performing inference (i.e. classification, segmentation and detection tasks) over images, including biomedical settings.
Despite recent efforts [5], [6], [7] in acquiring large labeled datasets – mainly for diagnosis, that is, classification tasks – most biomedical image domains suffer from a lack of labeled data. Therefore, it is highly desirable to acquire the most amount of knowledge possible with the few labeled data available in the literature, while also using the vast amounts of unlabeled data present in some domains. Aiming to lessen the requirements for labeled data in visual recognition, Domain Adaptation (DA) [8], [9], [10], [11] is the research area that comprises the theoretical background and methods for knowledge transfer between distinct tasks and/or data.
Chest X-rays (CXRs) are the most common type of radiological exam acquired nowadays, mainly due to their ability to aid in the detection and diagnosis of several kinds of ailments [6]. Several diverse health conditions such as pulmonary nodules [12], tuberculosis [13], pulmonary effusion, pneumonia and cardiomegaly [6], as well as bone fractures, can be assessed by CXRs in a quick and radiation-efficient exam. With the advent of large labeled datasets [6], [7], automation efforts have been proposed for aiding in the diagnosis of most of these illnesses. Computed Tomography (CT) exams yield volumetric images that allow physicians to perform 3D analysis of the thorax, but require more expensive hardware and submit the patient to between one and two orders of magnitude larger doses of radiation.
As argued by Zhang et al. [14] the understanding of anatomical objects in CXRs is useful for several clinical applications, such as pathological diagnosis, treatment evaluation, surgical planning and as an automated preprocessing step for Computer-Aided Detection (CAD) systems. Van Ginneken et al. [15], [16] point that the delineation of rib borders is part of this anatomical registration process whose automation can help physicians by providing a frame of reference for the location of abnormalities [14], [16] and help surgery planning. However, computerized analysis is still the largest beneficiary from algorithms for rib cage detection, being useful to mitigate both false positives [17] and false negatives [18] in nodule or rib fracture detection, which may also indicate other issues as damage to lung tissue, hemorrhage signs of osteoporosis or even an underlying cancer [19]. For instance, Austin et al. [18] report that between 82% and 95% of undetected lung cancers in CXRs were obscured by foreground bones such as ribs or clavicles. Yet, mainly due to the great burden involved in pixel-level annotations, there is only a tiny amount of publicly available labeled samples for the task of rib segmentation in CXRs, as further discussed in Section 3.3.
Given the problems related to acquiring pixel-level labels for rib segmentation and the usefulness of these data, the main contribution of this paper is a pipeline based on Conditional Domain Adaptation [20] for rib cage segmentation. Secondary contributions of this work include validating the use of volumetric data for CXR bone segmentation, presenting a new rib segmentation label set for the JSRT dataset [12], and defining a standard quantitative and qualitative comparison procedure for rib cage segmentation.
The remaining sections of this paper are organized as follows. Section 2 describes the current state of the literature on rib segmentation and suppression methods. Section 3 presents the proposed method for rib segmentation from CT data, while also discussing the experimental setup (i.e. datasets, metrics, etc) used in the tests. Section 4 shows the quantitative and qualitative results yielded by the experimental setup, comparing them to the baselines of rib segmentation and with pretrained DNNs. At last, Section 5 concludes the paper with our final remarks and future works.
Section snippets
Related work
This section presents the current literature on rib segmentation and suppression methods. It also describes the basis of Unsupervised Image-to-Image Translation, which is the theoretical basis for the proposed method.
Methodology
In this section, we present the proposed methodology for rib cage segmentation in CXRs by using Unsupervised Domain Adaptation (UDA) from DRRs. Our method leverages the capabilities of Conditional DA [20] to transfer the knowledge learned from synthetically flattened CT-scans to 2D CXRs. Sections 3.4 and 3.3 present a standardized set of metrics for rib segmentation evaluation and the datasets used in this research, respectively.
Results and discussion
Before discussing the rib cage segmentation results, we present a subset of segmentation predictions by CoDALungs in DRRs from LIDC-IDRI (Fig. 2), also showing the poor lung segmentation generalization achieved by the baselines. These intermediate results are important because inaccurate segmentation predictions on the DRRs should result in poor bone mask filtering for separating the ribs from other bones in the MIP outputs. Most segmentations from CoDALungs are near perfect in identifying lung
Conclusion
In this paper we presented a novel methodology for UDA applied to the problem of rib segmentation using Conditional Domain Adaptation [20]. The proposed pipeline uses higher dimensional 3D data to acquire two sets of flattened 2D images: DRRs that visually resemble real CXRs – serving as training samples for rib segmentation; and bone segmentation semantic maps that can be curated in order to become pixel-level rib segmentation labels.
We also proposed a novel evaluation procedure for rib
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
Authors would like to thank NVIDIA for the donation of the GPUs that allowed the execution of all experiments in this paper. We also thank CAPES, CNPq (424700/2018-2 and 311395/2018-0), and FAPEMIG (APQ-00449-17 and APQ-00519-20 -- CAD-COVID-19 Project) for the financial support provided for this research.
References (39)
- et al.
A survey on deep learning in medical image analysis
Med. Image Anal.
(2017) - et al.
Deep visual domain adaptation: a survey
Neurocomputing
(2018) - et al.
Atlas-based rib-bone detection in chest X-rays
Comput. Med. Imaging Graph.
(2016) - et al.
Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database
Med. Image Anal.
(2006) - et al.
ImageNet classification with deep convolutional neural networks
- et al.
U-net: convolutional networks for biomedical image segmentation
International Conference on Medical Image Computing and Computer Assisted Intervention
(2015) - et al.
Faster R-CNN: towards real-time object detection with region proposal networks
Advances in Neural Information Processing Systems
(2017) - et al.
Preparing a collection of radiology examinations for distribution and retrieval
J. Am. Med. Inform. Assoc.
(2015) - et al.
ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases
Conference on Computer Vision and Pattern Recognition
(2017) - A. Bustos, A. Pertusa, J.-M. Salinas, M. de la Iglesia-Vayá, PadChest: A Large Chest X-ray Image Dataset with...
Visual domain adaptation: a survey of recent advances
IEEE Signal Process. Mag.
Transfer learning for visual categorization: asurvey
IEEE Trans. Neural Netw. Learn. Syst.
Development of a digital image database for chest radiographs with and without a lung nodule receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules
Am. J. Roentgenol.
Two public chest X-ray datasets for computer-aided screening of pulmonary diseases
Quant. Imaging Med. Surg.
Task driven generative modeling for unsupervised domain adaptation: application to X-ray image segmentation
International Conference on Medical Image Computing and Computer Assisted Intervention
Automatic delineation of ribs in frontal chest radiographs
Medical Imaging 2000: Image Processing
Computer-aided diagnosis in chest radiography: asurvey
IEEE Trans. Med. Imaging
Fully automatic lung segmentation and rib suppression methods to improve nodule detection in chest radiographs
J. Med. Signals Sens.
Cited by (0)
- ☆
Handle by Associate Editor-in-Chief Gabriella Sanniti di Baja, PhD.