Elsevier

Pattern Recognition Letters

Volume 93, 1 July 2017, Pages 162-171
Pattern Recognition Letters

Analyzing the evolution of breast tumors through flow fields and strain tensors

https://doi.org/10.1016/j.patrec.2016.11.003Get rights and content

Highlights

  • We propose a method to quantify and visualize the changes of breast tumors.

  • Flow fields and strain tensors were used in this method.

  • We used OWA operators to aggregate robust optical flow methods.

  • The proposed method provides a good quantification and visualization of breast tumor.

Abstract

Breast cancer is one of the most perilous diseases that annually attack thousands of women. Physicians usually monitor the breast tumor changes during the course of a chemotherapy treatment. Computer programs may help physicians to predict the pathological response in order to adjust the medical treatment to produce the intended effects. This paper proposes a method for quantifying and visualizing the changes of breast tumors of cases undergoing medical treatment through strain tensors. The proposed method determines the displacement fields between each follow-up mammogram and its baseline. To compute the displacement fields, we evaluated the performance of eight robust and recent optical flow methods through landmark-based error and statistical analysis. Since, there is no ground truth to evaluate the optical flow methods when they are applied to mammograms, we propose to aggregate the best optical flow methods using ordered weighted averaging operators. The aggregated optical flow methods using the ‘as many as possible’ operator yields the smallest landmark-based error among three aggregation approaches analyzed with the proposed algorithm. The resulting optical flow is then used to estimate the strain tensors. The proposed method provides a good quantification and visualization for breast tumor changes and that helps physicians to plan treatment for their patients.

Introduction

Breast cancer is one of the most dangerous diseases that attacks humans, and yet it attacks women more than men [15]. Breast cancer is a group of cancer cells that can grow larger in breast tissues and may move to other areas of the human body. Mammographies are an effective method for detecting breast cancer in its early stage. Mammographic images are X-ray images of the breast. They are usually captured from different mammographic views. The most common ones are craniocaudal (CC) and mediolateral oblique (MLO). Physicians usually monitor the breast tumor changes of their patients during the course of chemotherapy, and they attempt to predict pathological response in order to adjust the treatment to produce the intended effects. Computer-aided diagnosis (CAD) systems help physicians to diagnose and follow-up their cases. In this paper, we propose a computer method for quantifying and visualizing breast tumor changes in follow-up (temporal) mammograms. Optical flow models with strain tensors are proposed to make this analysis.

In the literature, several studies have been carried out to detect the changes between two successive medical images of the same subject (breast, lung, heart, etc.). Most are based on computing optical flow. Given two successive images for the same view, an optical flow algorithm computes the flow fields (i.e., displacement vectors), which map all pixels from the first image onto their new positions in the second image. Indeed, several optical flow methods have been used in medical image analysis. Bhat and Liebling [5] used the Lucas-Kanade (LK) method to separate the bright-field microscopy image sequence of the beating embryonic heart into two image sequences, which were then analyzed to characterize the motion of the blood and heart-wall separately. In [20], [27], six deformable registration algorithms were compared to quantify tumor changes during neoadjuvant chemotherapy. The authors used Jacobian determinants and intensity residual color map to visualize the changes. Teo and Pistorius [26] investigated the feasibility and accuracy of tracking the motion of an intruding organ-at-risk at the edges of a treatment field using a local optical flow analysis of electronic portal images. Antink et al. [4] used a modified version of the LK method to estimate the flow fields between thoracic 4D computed tomography sequences. In [25], a B-spline based deformable image registration algorithm was used to align two sets of computed tomography (CT) image acquired pre- and post-treatment. The B-spline difference maps generated from the registered CT images show the regions related to the expansion or shrinkage of the metastatic tumors. Krueger et al. [12] used an optical flow method to estimate the displacement of pre- and post-compression ultrasound images. Lee [13] used optical flow methods to align mammograms and then they compared and evaluated them to detect abnormalities.

In general, the main limitation of using optical flow methods with medical images is the lack of a ground truth for assessing their accuracy. To solve this problem, we propose using a set of robust optical flow methods with mammograms and then aggregate the best ones. This yields more confident results, since we combine the merits of these optical flow methods. In this paper, we use ordered weighted averaging aggregation (OWA) operators to aggregate the results of the optical flow methods.

A preliminary work [2] presented the basic idea of this study. However, the current paper contains more contributions, such as using more robust optical flow methods, proposing an aggregation approach of flow fields and adding more analysis, experiments and discussion. This paper proposes a method to quantify and visualize breast tumor changes for women undergoing chemotherapy treatment. The inputs of the proposed system are two mammograms for the same breast, one before the treatment (baseline mammogram) and one after the treatment (follow-up mammogram). To estimate the strain tensors, the system applies five successive stages to the input mammograms (see Fig. 1). Firstly, five preprocessing operations are applied. Secondly, the optical flow between mammograms is computed. In this paper, we assess eight robust optical flow methods. Thirdly, an OWA aggregation approach [28] is used to aggregate the results of the best optical flow methods. The aggregated flow fields are then used to calculate the strain tensors. Finally, strain tensors are shown to physicians to examine tumor changes. A negative strain denotes the decrease in distance between two reference points, and thus indicates a regional shrinkage. In contrast, a positive strain refers to a regional expansion. The proposed system produces color codes that help to visualize breast tumor changes. The main contributions of this paper are:

  • To the best of our knowledge, this is the first method for analyzing the evolution of breast tumors in temporal mammograms using optical flow and strain tensors.

  • The main limitation of using optical flow methods with medical images is the lack of a ground truth to assess their accuracy. To cope with the aforementioned problem, we propose to use a set of robust optical flow methods with mammograms and then aggregate the best ones using OWA operators. The optical flow aggregation step is also a clear contribution of this paper.

  • The proposed method quantifies and visualizes the evolution of breast tumors in follow-up (temporal) mammograms. To evaluate the proposed method, we need temporal mammograms for each patient (i.e., mammograms for the same patient acquired at different times). Indeed, there are very few public mammographic images datasets: mini-MIAS, DDSM, INbreast and BancoWeb. These datasets do not include temporal mammograms, so we could not use any public dataset for evaluating the proposed method. As there is no public temporal mammographic images dataset, we have collected temporal mammograms for several patients from the Hospital Universitari Sant Joan (Reus, Spain). To help researchers make further contributions in analyzing the evolution of breast tumors, we plan to make this dataset publicly available.

  • A comparison with the related work is given in this paper to demonstrate the effectiveness of the proposed method.

The rest of this article is organized as follows. Section 2 explains each step of the proposed system. Section 3 presents the experimental results and discusses them. Section 4 summarizes this study and introduces some lines of future work.

Section snippets

Methods

Fig. 1 presents the five stages of the proposed system: pre-processing, computation of optical flow, aggregation, calculation of strain tensors and visualization. The subsections below give a detailed explanation of the first four steps of the proposed system.

Experimental results and discussion

This section presents the mammographic dataset, the evaluation methods and the experimental results of the proposed method.

Conclusion and future work

In this chapter we have proposed a computer method for quantifying and visualizing the changes in breast tumors for patients undergoing medical treatments using strain tensors. The proposed method consists of five successive stages: pre-processing, calculation of the optical flow, aggregation, calculation of strain tensors and visualization. We determined the displacement fields between each follow-up mammogram and its baseline. The displacement fields obtained were then used to calculate the

Acknowledgments

The authors thank Dr. Meritxell Arenas, Dr. Luis Martin and Dr. Anna Magarolas (Hospital Universitari Sant Joan, Reus, Spain) for their help in preparing the dataset used in this work. This research was partly supported by the Spanish Government through project TIN2012-37171-C02-02.

References (29)

  • J.-Y. Bouguet

    Pyramidal Implementation of the Affine Lucas Kanade Feature Tracker Description of the Algorithm

    (2001)
  • T. Brox et al.

    Large displacement optical flow: descriptor matching in variational motion estimation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • A. Bruhn et al.

    Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods

    Int. J. Comput. Vis.

    (2005)
  • R. Chandrasekhar et al.

    Gross segmentation of mammograms using a polynomial model

    Proceedings of the 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Bridging Disciplines for Biomedicine

    (1996)
  • Cited by (0)

    View full text