Elsevier

Medical Image Analysis

Volume 18, Issue 3, April 2014, Pages 500-514
Medical Image Analysis

Whole myocardium tracking in 2D-echocardiography in multiple orientations using a motion constrained level-set

https://doi.org/10.1016/j.media.2014.01.005Get rights and content

Highlights

  • We present a method to track the myocardium in 2D US sequences in any orientation.

  • We constrain the evolving contour to satisfy a level conservation hypothesis.

  • Robustness is obtained by adjusting the hyperparameters in an adaptive way.

  • Comparison is made with experts references from 900 images of clinical interest.

  • A complete study on the influence of the parameters settings is carried out.

Abstract

The segmentation and tracking of the myocardium in echocardiographic sequences is an important task for the diagnosis of heart disease. This task is difficult due to the inherent problems of echographic images (i.e. low contrast, speckle noise, signal dropout, presence of shadows). In this article, we extend a level-set method recently proposed in Dietenbeck et al. (2012) in order to track the whole myocardium in echocardiographic sequences. To this end, we enforce temporal coherence by adding a new motion prior energy to the existing framework. This motion prior term is expressed as new constraint that enforces the conservation of the levels of the implicit function along the image sequence. Moreover, the robustness of the proposed method is improved by adjusting the associated hyperparameters in a spatially adaptive way, using the available strong a priori about the echocardiographic regions to be segmented. The accuracy and robustness of the proposed method is evaluated by comparing the obtained segmentation with experts references and to another state-of-the-art method on a dataset of 15 sequences (≃ 900 images) acquired in three echocardiographic views. We show that the algorithm provides results that are consistent with the inter-observer variability and outperforms the state-of-the-art method. We also carry out a complete study on the influence of the parameters settings. The obtained results demonstrate the stability of our method according to those values.

Introduction

Clinical assessment of the left-ventricular function is essential for the diagnosis of heart diseases. Amongst other imaging techniques, ultrasound imaging is a popular tool since it allows real-time visualization of the heart motion through the cardiac cycle. However the extraction of functional parameters often requires a manual delineation of the heart boundaries by an expert cardiologist. The inherent problems linked to echocardiography (e.g. presence of speckle, signal dropouts) make this task prone to errors, subjective and time-consuming. In order to speed up the analysis and decrease the variability in the delineation, the automatic identification and tracking of the myocardial borders is thus a very active research area (Noble and Boukerroui, 2006, Casero and Noble, 2008, Nascimento and Marques, 2008, Carneiro et al., 2012). Despite these important research efforts, segmentation and tracking in echocardiography still face noticeable limitations: as shown in Noble and Boukerroui (2006), most of the studies deal with the segmentation of the endocardium only and operate in one particular echocardiographic view. As a consequence, very few papers concern the tracking of the whole myocardium (Dias and Leitao, 1996, Chalana et al., 1996, Zhou et al., 2004, Zhou et al., 2005) or of the endocardium in multiple views (Comaniciu et al., 2004, Casero and Noble, 2008). In this context, the main applicative originality of this study is the design of a methodology which allows to segment and track the whole myocardium (i.e. endo- and epicardium) in the main echocardiographic planes (i.e. parasternal short-axis, apical 4-chamber and apical 2-chamber views).

While in this work we focus on 2D segmentation, let us however note that epicardial segmentation has recently received more attention in 3D echocardiography. In Zhu et al. (2010), an incompressibility constraint was introduced in an active contour framework to segment the whole myocardium in 3D echography. In Orderud et al. (2008), coupled segmentation of endo- and epicardial borders in 3D echocardiography was performed by using a Kalman filter-based tracking framework. Although 3D imaging is receiving increasing attention, its use in clinical routine is still limited and usually coupled to 2D acquisitions which provide a better in-plane resolution. As compared to 3D, it should be noted that the segmentation and tracking of 2D echocardiographic sequences raise specific problems: the shape of the cardiac structures indeed varies considerably according to the selected echographic acquisition view, and through plane motion yields shape variations that may result in partial occlusion.

When segmenting sequences and tracking an object over time, the knowledge of the underlying motion may bring valuable information and help improving segmentation results and speed. Indeed parts of the object that are hidden in a frame might be visible in another one; temporal coherence implies that contours in successive frames have similar shapes. In the context of echocardiography, many techniques have been proposed to include the motion information in the tracking process: motion model (Chalana et al., 1996, Nascimento and Marques, 2008, Leung et al., 2011), dynamical shape priors (Bosch et al., 2002, Casero and Noble, 2008), Kalman filtering (Jacob et al., 1999, Comaniciu et al., 2004).

In Chalana et al. (1996), the authors proposed a coupled active contours technique based on snakes to segment the whole myocardium in parasternal short-axis sequences. In order to segment the myocardium over the whole sequence, the authors have suggested to minimize an energy composed of 2 terms: a gradient based term and a motion continuity term which enforces the contour to contract during systole and to expand during diastole. Nascimento and Marques (2008) presented an algorithm for the tracking of the endocardial boundary in apical 4-chamber sequences. The evolution of the shape and motion parameters is performed on an edge map using a bank of switched dynamic systems. To deal with possible outliers and multiple dynamics, a filtering algorithm was proposed, which propagates the a posteriori density of the unknown shape and motion parameters using a tree of probability data association filters. In the context of endocardial segmentation in 3D echocardiography, Leung et al. (2011) proposed a motion model based on PCA. After an alignment step of the dataset, Procrustes analysis is performed in order to obtain the inter-frame motion for all frames of all the sequences. The cardiac motion is modeled as an affine transformation and PCA is then applied on the affine transformation in order to learn the motion model. This PCA motion model is then introduced in the optical flow equation with the assumption that the PCA parameters are constant in small regions around the contour.

Jacob et al. (1999) incorporated shape PCA and Kalman filtering in an active contour framework. Hereto, PCA is applied to a dataset that describes heart shape variation for one specific acquisition view. The measurement step is performed using a combination of spatio-temporal noise reduction filtering and feature detection from phase information. Comaniciu et al. (2004) developed an information fusion framework to track the endocardium in parasternal short-axis and apical 4-chamber views. They formulated the tracking framework as an information fusion problem using Kalman filtering and strongly-adapted PCA and compute an uncertainty measure from the optical flow. This allows to discard flows estimated from uncertain areas such as drop-out regions while giving more importance to flows with high confidence. This technique was further extended to the tracking of the whole myocardium by Zhou et al., 2004, Zhou et al., 2005, where the two borders were modeled as a single point in the shape space. The coupled evolution of the contours proved to be more robust in the tracking process since more information is considered in the evolution process.

Bosch et al. (2002) used an adaptation of the active appearance model (AAM) approach referred to as the active appearance motion model (AAMM) to represent the shape and appearance of the endocardium, as well as its motion. However their method was only tested on apical 4-chamber views. Casero and Noble (2008) proposed a framework to take into account the cyclic dynamics of the heart shape. In this framework, PCA is applied on the dataset of sequences, where the pose parameters are estimated using a modified Procrustes alignment in order not to remove temporal variability inside a sequence. Temporal models of the whole myocardium in parasternal short-axis, apical 4-chamber, apical 2-chamber and apical 3-chamber views are then learned using this technique.

The main drawback of these techniques is that the motion knowledge is often learned via an interactive training process. Though this training process can take place off-line, it involves considerable effort and expertise. More importantly, since we are interested in tracking the myocardium in multiple orientations, it is difficult to make use of these approaches due to the complexity of the heart motion. It would thus require to learn a motion model per view as in Casero and Noble (2008). Furthermore, during the learning process, one has to deal with the tedious dilemma of considering both healthy and pathological subjects, since the generalization power of the statistical model depends ultimately on the database used in the training phase.

The following novelties are thus introduced in this paper.1 We extend the previously described level-set formalism (Dietenbeck et al., 2012) to the tracking of the whole myocardium in multiple orientations (i.e. parasternal short-axis, apical 4-chamber and apical 2-chamber views). To this end, we propose to constrain the evolving contour in order to satisfy a level conservation hypothesis. This assumption ensures that the zero-level of the implicit function evolves according to the underlying motion field throughout the cardiac sequence. We then express this constraint as a motion energy and include it into the variational framework described in Dietenbeck et al. (2012). The interest of such formulation stems from the fact that this constraint does not require any learning step since we are not using a motion model. Compared to previous works that also use this assumption (Papin et al., 2000, Unal et al., 2005), our formulation is set in an energy minimization process, thus ensuring the convergence of the algorithm to a minimum. Moreover, the motion being estimated prior to the tracking, this formulation allows to choose the motion estimation algorithm that provides the best results while keeping the same tracking algorithm. Furthermore, we demonstrate the importance of this energy term by comparing tracking results with and without this term showing an improvement of the results by 25% in terms of Modified Dice, Mean Absolute Distance and Hausdorff Distance (whose definitions are given in Section 6.1).

Another novelty introduced in this article concerns the setting of the hyperparameters. Indeed, most segmentation methods formally rely on different terms reflecting the different types of information driving the segmentation process (e.g. data attachment term, shape prior, motion model, etc.). Tuning the weighting of these terms represents a difficult task and is often done empirically and in a global way. However this implies that the same weights are applied over the whole contour, in regions presenting different properties (such as different contrast) where the influence of each parameter could be needed in a different way. To tackle this problem, we propose in this paper an original approach that makes use of a very stronga prioriabout the image properties based on the heart anatomy. In this context, we introduce spatially varying parameters that allows modeling the problems linked to the different image properties depending on the position along the myocardium. More specifically, we divide each frame according to the AHA nomenclature (Cerqueira et al., 2002) and apply different weighting in each segment in order to take into account regions where the data information can be trusted or in the contrary where the prior should preponderate. We then show that the use of such an anatomical prior for the hyperparameters makes the segmentation more adaptive and robust to hyperparameters changes.

The paper is organized as follows. In Section 2, we recall the general level-set framework and describe the energy functional that will be minimized. In Section 3, we detail the level conservation hypothesis and the corresponding energy term. In Section 4, we describe the setting of the hyperparameters and the image subdivision according to the anatomical properties of the sequence. Implementation issues are further discussed in Section 5. Section 6 is devoted to the results obtained from echocardiographic clinical sequences. More specifically, we compared our results with experts references and a recent method proposed by Hamou and El-Sakka (2010). We also evaluate the robustness of our algorithm with respect to the hyperparameters. We also show the generic nature of our method by applying it on 10 cardiac cine-MRI sequences and compared the obtained results with the references from one expert. The main conclusions and perspectives are given in Section 7.

Section snippets

Context

In this Section, we recall the level-set framework and the method we recently proposed for the segmentation of 2D echocardiographic images and describe the algorithm used for the motion estimation. Level-sets correspond to a class of deformable models where the shape to be recovered is captured by propagating an interface represented by the zero level-set of a smooth function which is usually called the level-set function. The evolution of the interface is generally derived through a

Motion term

In this Section, we describe how we use the motion information to guide the evolution of the active contour. Previous studies have generally dealt with this approach in two different ways: either considering motion as a data or as a prior knowledge.

When considering motion as a data, one can either estimate the motion prior to the segmentation (Papin et al., 2000, Unal et al., 2005, Herbulot et al., 2006) or perform a joint motion estimation and segmentation (Cremers and Soatto, 2005, Brox et

Spatially varying hyperparameters

A common problem of active contour methods is the setting of the hyperparameters weighting the different energy terms and their evaluation. Indeed, their values are often chosen empirically in order to give the same importance to all the terms or in the contrary to let one term preponderate over all the others. However, in echocardiography, it is interesting to vary the parameter influence according to some image information. Indeed in regions with a good contrast (e.g. area 1 in Fig. 1(c),

Level-set evolution

The implicit function is represented by a signed distance function ϕ and is re-initialized every iteration using a fast marching scheme (Sussman et al., 1998). In order to improve efficiency, we only compute values of ϕ in a narrow band around the zero level-set. The neighborhood B(p) defining the localization of the data attachment term is chosen in our case as a circular neighborhood, with radius RN chosen as the average half thickness of the myocardium, i.e. 8 pixels in our case. In the same

Experimental data

The reference dataset is composed of 15 echocardiographic sequences (5 per view) acquired from 11 healthy volunteers. The sequences were recorded using a GE Vivid E9 system equipped with a 2.5 MHz M5S probe (GE Vingmed Ultrasound, Horten, Norway). Two experts manually outlined the myocardium on one cardiac cycle per sequence resulting in the following reference distribution:

  • 290 frames in parasternal short-axis view,

  • 300 frames in apical 4-chamber view,

  • 300 frames in apical 2-chamber view.

Error measures

To

Conclusion

In this article, we have described a new motion prior energy that when minimized imposes a level consistency to the level-set function. This energy is then added to a recently proposed framework for the segmentation and tracking of the whole myocardium in multiple orientations. We have also proposed to take advantage of the anatomical and image properties of echocardiographic data to adjust the hyperparameters spatially in order to make the method more robust. The algorithm is then evaluated on

Acknowledgment

This work was partially supported by Agence Nationale de la Recherche (US-Tagging Grant), the region Rhône-Alpes (ExploraDoc and AccueilDoc Grants) and Egide (PHC Tournesol Grant). This work was conducted in the framework of the LabEX PRIMES (“Physics Radiobiology Medical Imaging and Simulation”).

References (49)

  • M. Alessandrini et al.

    Myocardial motion estimation from medical images using the monogenic signal

    IEEE Trans. Image Process.

    (2013)
  • Berbari, R.E., Bloch, I., Redheuil, A., Angelini, E., E.Mousseaux, Frouin, F., Herment, A., 2007. An automated...
  • J. Bosch et al.

    Automatic segmentation of echocardiographic sequences by active appearance motion models

    IEEE Trans. Med. Imag.

    (2002)
  • Brox, T., Bruhn, A., Weickert, J., 2006. Variational motion segmentation with level sets. In: European Conference on...
  • T. Brox et al.

    Combined region and motion-based 3d tracking of rigid and articulated objects

    IEEE Trans. Pattern Anal. Machine Intell.

    (2010)
  • Bruhn, A., Weickert, J., 2006. Confidence measures for variational optic flow methods. In: Geometric Properties from...
  • G. Carneiro et al.

    The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods

    IEEE Trans. Image Process.

    (2012)
  • Casero, R., Noble, J., 2008. A novel explicit 2d+t cyclic shape model applied to echocardiography. In: Medical Image...
  • M. Cerqueira et al.

    Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart

    Circulation

    (2002)
  • V. Chalana et al.

    A methodology for evaluation of boundary detection algorithms on medical images

    IEEE Trans. Med. Imag.

    (1997)
  • V. Chalana et al.

    A multiple active contour model for cardiac boundary detection on echocardiographic sequences

    IEEE Trans. Med. Imag.

    (1996)
  • T. Chan et al.

    Active contours without edges

    IEEE Trans. Image Process.

    (2001)
  • D. Comaniciu et al.

    Robust real-time myocardial border tracking for echocardiography: an information fusion approach

    IEEE Trans. Med. Imag.

    (2004)
  • D. Cremers

    Dynamical statistical shape priors for level set-based tracking

    IEEE Trans. Pattern Anal. Machine Intell.

    (2006)
  • Cited by (17)

    • Graph cut-based method for segmenting the left ventricle from MRI or echocardiographic images

      2017, Computerized Medical Imaging and Graphics
      Citation Excerpt :

      Another issue with active contours with echocardiographic images is that they can “bleed” in areas suffering from a signal dropout. To overcome this, authors often use shape priors (Dietenbeck et al., 2014, 2012; Pluempitiwiriyawej et al., 2005). While they do work, methods including a shape prior are prohibitively slow as well as mathematically and algorithmically complex.

    • A novel myocardium segmentation approach based on neutrosophic active contour model

      2017, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      Computer aided diagnosis (CAD) system has been developed to identify the infarcted myocardium [9]. Many approaches have been proposed to identify and track myocardial borders [10], the ventricular cavity (i.e. endocardium) [11], epicardial boundary [12], and the full myocardium using echocardiographic images [13,14]. Ziwirn et al. [8] presented an endocardial boundary (inner boundary) detection method in myocardial images characterized by low signal-to-noise ratios.

    • Estimation of Large-Scale Organ Motion in B-Mode Ultrasound Image Sequences: A Survey

      2015, Ultrasound in Medicine and Biology
      Citation Excerpt :

      Common error measures are the mean squared error (MSE), the root mean square error (RMSE) and mean absolute difference (MAD). In the case of contours, the Hausdorff distance (Dietenbeck et al. 2014; Nascimento and Marques 2008) and the point-to-surface distance (Wang et al. 2010; Yang et al. 2008) have been used. Statistical tests include linear regression (Jacob et al. 2002; Li et al. 2013; Veronesi et al. 2006; Zahnd et al. 2013) and the Bland–Altman test (Duan et al. 2009; Jacob et al. 2002; Veronesi et al. 2006) and, for regions, Pearson's correlation coefficient test (Chalana et al. 1996; Mikic et al. 1998; Stoitsis et al. 2005).

    • Analytic signal phase-based myocardial motion estimation in tagged MRI sequences by a bilinear model and motion compensation

      2015, Medical Image Analysis
      Citation Excerpt :

      In that work, a coarse-to-fine B-spline scheme allows for the effective and robust computation of the displacement, and also a pyramidal refinement scheme helps to deal with large motions. From the myocardial motion field, various further studies have been realized in order to extract motion-related information, such as myocardium segmentation (Dietenbeck et al., 2014), local deformation (Kar et al., 2014; Oubel et al., 2012), and local region tracking (Arif et al., 2014; Luo et al., 2014; Sun et al., 2011). For these information extraction methods, the key point is to estimate an accurate myocardium motion field.

    View all citing articles on Scopus
    View full text