Analyzing the evolution of breast tumors through flow fields and strain tensors
Graphical abstract
Introduction
Breast cancer is one of the most dangerous diseases that attacks humans, and yet it attacks women more than men [15]. Breast cancer is a group of cancer cells that can grow larger in breast tissues and may move to other areas of the human body. Mammographies are an effective method for detecting breast cancer in its early stage. Mammographic images are X-ray images of the breast. They are usually captured from different mammographic views. The most common ones are craniocaudal (CC) and mediolateral oblique (MLO). Physicians usually monitor the breast tumor changes of their patients during the course of chemotherapy, and they attempt to predict pathological response in order to adjust the treatment to produce the intended effects. Computer-aided diagnosis (CAD) systems help physicians to diagnose and follow-up their cases. In this paper, we propose a computer method for quantifying and visualizing breast tumor changes in follow-up (temporal) mammograms. Optical flow models with strain tensors are proposed to make this analysis.
In the literature, several studies have been carried out to detect the changes between two successive medical images of the same subject (breast, lung, heart, etc.). Most are based on computing optical flow. Given two successive images for the same view, an optical flow algorithm computes the flow fields (i.e., displacement vectors), which map all pixels from the first image onto their new positions in the second image. Indeed, several optical flow methods have been used in medical image analysis. Bhat and Liebling [5] used the Lucas-Kanade (LK) method to separate the bright-field microscopy image sequence of the beating embryonic heart into two image sequences, which were then analyzed to characterize the motion of the blood and heart-wall separately. In [20], [27], six deformable registration algorithms were compared to quantify tumor changes during neoadjuvant chemotherapy. The authors used Jacobian determinants and intensity residual color map to visualize the changes. Teo and Pistorius [26] investigated the feasibility and accuracy of tracking the motion of an intruding organ-at-risk at the edges of a treatment field using a local optical flow analysis of electronic portal images. Antink et al. [4] used a modified version of the LK method to estimate the flow fields between thoracic 4D computed tomography sequences. In [25], a B-spline based deformable image registration algorithm was used to align two sets of computed tomography (CT) image acquired pre- and post-treatment. The B-spline difference maps generated from the registered CT images show the regions related to the expansion or shrinkage of the metastatic tumors. Krueger et al. [12] used an optical flow method to estimate the displacement of pre- and post-compression ultrasound images. Lee [13] used optical flow methods to align mammograms and then they compared and evaluated them to detect abnormalities.
In general, the main limitation of using optical flow methods with medical images is the lack of a ground truth for assessing their accuracy. To solve this problem, we propose using a set of robust optical flow methods with mammograms and then aggregate the best ones. This yields more confident results, since we combine the merits of these optical flow methods. In this paper, we use ordered weighted averaging aggregation (OWA) operators to aggregate the results of the optical flow methods.
A preliminary work [2] presented the basic idea of this study. However, the current paper contains more contributions, such as using more robust optical flow methods, proposing an aggregation approach of flow fields and adding more analysis, experiments and discussion. This paper proposes a method to quantify and visualize breast tumor changes for women undergoing chemotherapy treatment. The inputs of the proposed system are two mammograms for the same breast, one before the treatment (baseline mammogram) and one after the treatment (follow-up mammogram). To estimate the strain tensors, the system applies five successive stages to the input mammograms (see Fig. 1). Firstly, five preprocessing operations are applied. Secondly, the optical flow between mammograms is computed. In this paper, we assess eight robust optical flow methods. Thirdly, an OWA aggregation approach [28] is used to aggregate the results of the best optical flow methods. The aggregated flow fields are then used to calculate the strain tensors. Finally, strain tensors are shown to physicians to examine tumor changes. A negative strain denotes the decrease in distance between two reference points, and thus indicates a regional shrinkage. In contrast, a positive strain refers to a regional expansion. The proposed system produces color codes that help to visualize breast tumor changes. The main contributions of this paper are:
- •
To the best of our knowledge, this is the first method for analyzing the evolution of breast tumors in temporal mammograms using optical flow and strain tensors.
- •
The main limitation of using optical flow methods with medical images is the lack of a ground truth to assess their accuracy. To cope with the aforementioned problem, we propose to use a set of robust optical flow methods with mammograms and then aggregate the best ones using OWA operators. The optical flow aggregation step is also a clear contribution of this paper.
- •
The proposed method quantifies and visualizes the evolution of breast tumors in follow-up (temporal) mammograms. To evaluate the proposed method, we need temporal mammograms for each patient (i.e., mammograms for the same patient acquired at different times). Indeed, there are very few public mammographic images datasets: mini-MIAS, DDSM, INbreast and BancoWeb. These datasets do not include temporal mammograms, so we could not use any public dataset for evaluating the proposed method. As there is no public temporal mammographic images dataset, we have collected temporal mammograms for several patients from the Hospital Universitari Sant Joan (Reus, Spain). To help researchers make further contributions in analyzing the evolution of breast tumors, we plan to make this dataset publicly available.
- •
A comparison with the related work is given in this paper to demonstrate the effectiveness of the proposed method.
The rest of this article is organized as follows. Section 2 explains each step of the proposed system. Section 3 presents the experimental results and discusses them. Section 4 summarizes this study and introduces some lines of future work.
Section snippets
Methods
Fig. 1 presents the five stages of the proposed system: pre-processing, computation of optical flow, aggregation, calculation of strain tensors and visualization. The subsections below give a detailed explanation of the first four steps of the proposed system.
Experimental results and discussion
This section presents the mammographic dataset, the evaluation methods and the experimental results of the proposed method.
Conclusion and future work
In this chapter we have proposed a computer method for quantifying and visualizing the changes in breast tumors for patients undergoing medical treatments using strain tensors. The proposed method consists of five successive stages: pre-processing, calculation of the optical flow, aggregation, calculation of strain tensors and visualization. We determined the displacement fields between each follow-up mammogram and its baseline. The displacement fields obtained were then used to calculate the
Acknowledgments
The authors thank Dr. Meritxell Arenas, Dr. Luis Martin and Dr. Anna Magarolas (Hospital Universitari Sant Joan, Reus, Spain) for their help in preparing the dataset used in this work. This research was partly supported by the Spanish Government through project TIN2012-37171-C02-02.
References (29)
- et al.
Direct three-dimensional myocardial strain tensor quantification and tracking using zharp
Med. Image Anal.
(2008) - et al.
Temporal mammogram image registration using optimized curvilinear coordinates
Comput. Methods Programs Biomed.
(2016) - et al.
Determining optical flow
Artif. Intell.
(1981) - et al.
European cancer mortality predictions for the year 2015: does lung cancer have the highest death rate in eu women?
Ann. Oncol.
(2015) - et al.
Dramms: deformable registration via attribute matching and mutual-saliency weighting
Med. Image Anal.
(2011) - et al.
Automated assessment of breast tissue density in digital mammograms
Comput. Vision Image Understanding
(2010) - et al.
Detail-preserving median based filters in image processing
Pattern Recognit. Lett.
(1994) - et al.
Analysis of the evolution of breast tumours using strain tensors
Artificial Intelligence Research and Development - Proceedings of the 18th International Conference of the Catalan Association for Artificial Intelligence, Valencia, Catalonia, Spain, October 21–23, 2015.
(2015) - et al.
Evaluation of advanced Lukas–Kanade optical flow on thoracic 4D-CT
J. Clin. Monit. Comput.
(2013) - et al.
Cardiac tissue and erythrocyte separation in bright-field microscopy images of the embryonic zebrafish heart for motion estimation
Proceedings of the 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA, June 28 - July 1, 2009
(2009)