Whole myocardium tracking in 2D-echocardiography in multiple orientations using a motion constrained level-set
Graphical abstract
Introduction
Clinical assessment of the left-ventricular function is essential for the diagnosis of heart diseases. Amongst other imaging techniques, ultrasound imaging is a popular tool since it allows real-time visualization of the heart motion through the cardiac cycle. However the extraction of functional parameters often requires a manual delineation of the heart boundaries by an expert cardiologist. The inherent problems linked to echocardiography (e.g. presence of speckle, signal dropouts) make this task prone to errors, subjective and time-consuming. In order to speed up the analysis and decrease the variability in the delineation, the automatic identification and tracking of the myocardial borders is thus a very active research area (Noble and Boukerroui, 2006, Casero and Noble, 2008, Nascimento and Marques, 2008, Carneiro et al., 2012). Despite these important research efforts, segmentation and tracking in echocardiography still face noticeable limitations: as shown in Noble and Boukerroui (2006), most of the studies deal with the segmentation of the endocardium only and operate in one particular echocardiographic view. As a consequence, very few papers concern the tracking of the whole myocardium (Dias and Leitao, 1996, Chalana et al., 1996, Zhou et al., 2004, Zhou et al., 2005) or of the endocardium in multiple views (Comaniciu et al., 2004, Casero and Noble, 2008). In this context, the main applicative originality of this study is the design of a methodology which allows to segment and track the whole myocardium (i.e. endo- and epicardium) in the main echocardiographic planes (i.e. parasternal short-axis, apical 4-chamber and apical 2-chamber views).
While in this work we focus on 2D segmentation, let us however note that epicardial segmentation has recently received more attention in 3D echocardiography. In Zhu et al. (2010), an incompressibility constraint was introduced in an active contour framework to segment the whole myocardium in 3D echography. In Orderud et al. (2008), coupled segmentation of endo- and epicardial borders in 3D echocardiography was performed by using a Kalman filter-based tracking framework. Although 3D imaging is receiving increasing attention, its use in clinical routine is still limited and usually coupled to 2D acquisitions which provide a better in-plane resolution. As compared to 3D, it should be noted that the segmentation and tracking of 2D echocardiographic sequences raise specific problems: the shape of the cardiac structures indeed varies considerably according to the selected echographic acquisition view, and through plane motion yields shape variations that may result in partial occlusion.
When segmenting sequences and tracking an object over time, the knowledge of the underlying motion may bring valuable information and help improving segmentation results and speed. Indeed parts of the object that are hidden in a frame might be visible in another one; temporal coherence implies that contours in successive frames have similar shapes. In the context of echocardiography, many techniques have been proposed to include the motion information in the tracking process: motion model (Chalana et al., 1996, Nascimento and Marques, 2008, Leung et al., 2011), dynamical shape priors (Bosch et al., 2002, Casero and Noble, 2008), Kalman filtering (Jacob et al., 1999, Comaniciu et al., 2004).
In Chalana et al. (1996), the authors proposed a coupled active contours technique based on snakes to segment the whole myocardium in parasternal short-axis sequences. In order to segment the myocardium over the whole sequence, the authors have suggested to minimize an energy composed of 2 terms: a gradient based term and a motion continuity term which enforces the contour to contract during systole and to expand during diastole. Nascimento and Marques (2008) presented an algorithm for the tracking of the endocardial boundary in apical 4-chamber sequences. The evolution of the shape and motion parameters is performed on an edge map using a bank of switched dynamic systems. To deal with possible outliers and multiple dynamics, a filtering algorithm was proposed, which propagates the a posteriori density of the unknown shape and motion parameters using a tree of probability data association filters. In the context of endocardial segmentation in 3D echocardiography, Leung et al. (2011) proposed a motion model based on PCA. After an alignment step of the dataset, Procrustes analysis is performed in order to obtain the inter-frame motion for all frames of all the sequences. The cardiac motion is modeled as an affine transformation and PCA is then applied on the affine transformation in order to learn the motion model. This PCA motion model is then introduced in the optical flow equation with the assumption that the PCA parameters are constant in small regions around the contour.
Jacob et al. (1999) incorporated shape PCA and Kalman filtering in an active contour framework. Hereto, PCA is applied to a dataset that describes heart shape variation for one specific acquisition view. The measurement step is performed using a combination of spatio-temporal noise reduction filtering and feature detection from phase information. Comaniciu et al. (2004) developed an information fusion framework to track the endocardium in parasternal short-axis and apical 4-chamber views. They formulated the tracking framework as an information fusion problem using Kalman filtering and strongly-adapted PCA and compute an uncertainty measure from the optical flow. This allows to discard flows estimated from uncertain areas such as drop-out regions while giving more importance to flows with high confidence. This technique was further extended to the tracking of the whole myocardium by Zhou et al., 2004, Zhou et al., 2005, where the two borders were modeled as a single point in the shape space. The coupled evolution of the contours proved to be more robust in the tracking process since more information is considered in the evolution process.
Bosch et al. (2002) used an adaptation of the active appearance model (AAM) approach referred to as the active appearance motion model (AAMM) to represent the shape and appearance of the endocardium, as well as its motion. However their method was only tested on apical 4-chamber views. Casero and Noble (2008) proposed a framework to take into account the cyclic dynamics of the heart shape. In this framework, PCA is applied on the dataset of sequences, where the pose parameters are estimated using a modified Procrustes alignment in order not to remove temporal variability inside a sequence. Temporal models of the whole myocardium in parasternal short-axis, apical 4-chamber, apical 2-chamber and apical 3-chamber views are then learned using this technique.
The main drawback of these techniques is that the motion knowledge is often learned via an interactive training process. Though this training process can take place off-line, it involves considerable effort and expertise. More importantly, since we are interested in tracking the myocardium in multiple orientations, it is difficult to make use of these approaches due to the complexity of the heart motion. It would thus require to learn a motion model per view as in Casero and Noble (2008). Furthermore, during the learning process, one has to deal with the tedious dilemma of considering both healthy and pathological subjects, since the generalization power of the statistical model depends ultimately on the database used in the training phase.
The following novelties are thus introduced in this paper.1 We extend the previously described level-set formalism (Dietenbeck et al., 2012) to the tracking of the whole myocardium in multiple orientations (i.e. parasternal short-axis, apical 4-chamber and apical 2-chamber views). To this end, we propose to constrain the evolving contour in order to satisfy a level conservation hypothesis. This assumption ensures that the zero-level of the implicit function evolves according to the underlying motion field throughout the cardiac sequence. We then express this constraint as a motion energy and include it into the variational framework described in Dietenbeck et al. (2012). The interest of such formulation stems from the fact that this constraint does not require any learning step since we are not using a motion model. Compared to previous works that also use this assumption (Papin et al., 2000, Unal et al., 2005), our formulation is set in an energy minimization process, thus ensuring the convergence of the algorithm to a minimum. Moreover, the motion being estimated prior to the tracking, this formulation allows to choose the motion estimation algorithm that provides the best results while keeping the same tracking algorithm. Furthermore, we demonstrate the importance of this energy term by comparing tracking results with and without this term showing an improvement of the results by 25% in terms of Modified Dice, Mean Absolute Distance and Hausdorff Distance (whose definitions are given in Section 6.1).
Another novelty introduced in this article concerns the setting of the hyperparameters. Indeed, most segmentation methods formally rely on different terms reflecting the different types of information driving the segmentation process (e.g. data attachment term, shape prior, motion model, etc.). Tuning the weighting of these terms represents a difficult task and is often done empirically and in a global way. However this implies that the same weights are applied over the whole contour, in regions presenting different properties (such as different contrast) where the influence of each parameter could be needed in a different way. To tackle this problem, we propose in this paper an original approach that makes use of a very stronga prioriabout the image properties based on the heart anatomy. In this context, we introduce spatially varying parameters that allows modeling the problems linked to the different image properties depending on the position along the myocardium. More specifically, we divide each frame according to the AHA nomenclature (Cerqueira et al., 2002) and apply different weighting in each segment in order to take into account regions where the data information can be trusted or in the contrary where the prior should preponderate. We then show that the use of such an anatomical prior for the hyperparameters makes the segmentation more adaptive and robust to hyperparameters changes.
The paper is organized as follows. In Section 2, we recall the general level-set framework and describe the energy functional that will be minimized. In Section 3, we detail the level conservation hypothesis and the corresponding energy term. In Section 4, we describe the setting of the hyperparameters and the image subdivision according to the anatomical properties of the sequence. Implementation issues are further discussed in Section 5. Section 6 is devoted to the results obtained from echocardiographic clinical sequences. More specifically, we compared our results with experts references and a recent method proposed by Hamou and El-Sakka (2010). We also evaluate the robustness of our algorithm with respect to the hyperparameters. We also show the generic nature of our method by applying it on 10 cardiac cine-MRI sequences and compared the obtained results with the references from one expert. The main conclusions and perspectives are given in Section 7.
Section snippets
Context
In this Section, we recall the level-set framework and the method we recently proposed for the segmentation of 2D echocardiographic images and describe the algorithm used for the motion estimation. Level-sets correspond to a class of deformable models where the shape to be recovered is captured by propagating an interface represented by the zero level-set of a smooth function which is usually called the level-set function. The evolution of the interface is generally derived through a
Motion term
In this Section, we describe how we use the motion information to guide the evolution of the active contour. Previous studies have generally dealt with this approach in two different ways: either considering motion as a data or as a prior knowledge.
When considering motion as a data, one can either estimate the motion prior to the segmentation (Papin et al., 2000, Unal et al., 2005, Herbulot et al., 2006) or perform a joint motion estimation and segmentation (Cremers and Soatto, 2005, Brox et
Spatially varying hyperparameters
A common problem of active contour methods is the setting of the hyperparameters weighting the different energy terms and their evaluation. Indeed, their values are often chosen empirically in order to give the same importance to all the terms or in the contrary to let one term preponderate over all the others. However, in echocardiography, it is interesting to vary the parameter influence according to some image information. Indeed in regions with a good contrast (e.g. area 1 in Fig. 1(c),
Level-set evolution
The implicit function is represented by a signed distance function and is re-initialized every iteration using a fast marching scheme (Sussman et al., 1998). In order to improve efficiency, we only compute values of in a narrow band around the zero level-set. The neighborhood defining the localization of the data attachment term is chosen in our case as a circular neighborhood, with radius chosen as the average half thickness of the myocardium, i.e. 8 pixels in our case. In the same
Experimental data
The reference dataset is composed of 15 echocardiographic sequences (5 per view) acquired from 11 healthy volunteers. The sequences were recorded using a GE Vivid E9 system equipped with a 2.5 MHz M5S probe (GE Vingmed Ultrasound, Horten, Norway). Two experts manually outlined the myocardium on one cardiac cycle per sequence resulting in the following reference distribution:
- •
290 frames in parasternal short-axis view,
- •
300 frames in apical 4-chamber view,
- •
300 frames in apical 2-chamber view.
Error measures
To
Conclusion
In this article, we have described a new motion prior energy that when minimized imposes a level consistency to the level-set function. This energy is then added to a recently proposed framework for the segmentation and tracking of the whole myocardium in multiple orientations. We have also proposed to take advantage of the anatomical and image properties of echocardiographic data to adjust the hyperparameters spatially in order to make the method more robust. The algorithm is then evaluated on
Acknowledgment
This work was partially supported by Agence Nationale de la Recherche (US-Tagging Grant), the region Rhône-Alpes (ExploraDoc and AccueilDoc Grants) and Egide (PHC Tournesol Grant). This work was conducted in the framework of the LabEX PRIMES (“Physics Radiobiology Medical Imaging and Simulation”).
References (49)
- et al.
Detection of the whole myocardium in 2d-echocardiography for multiple orientations using a geometrically constrained level-set
Med. Image Anal.
(2012) - et al.
Real-time segmentation by active geometric functions
Comput. Methods Programs Biomed.
(2010) - et al.
Evaluating a robust contour tracker on echocardiographic sequences
Med. Image Anal.
(1999) - et al.
Left ventricular border tracking using cardiac motion models and optical flow
Ultrasound Med. Biol.
(2011) - et al.
Automatic segmentation of the left ventricle cavity and myocardium in MRI data
Comput. Biol. Med.
(2006) - et al.
Left-ventricle myocardium segmentation using a coupled level-set with a priori knowledge
Comput. Med. Imag. Graph.
(2006) - et al.
Fronts propagating with curvature-dependent speed: algorithms based on Hamilton–Jacobi formulation
J. Comp. Phys.
(1988) - et al.
A review of segmentation methods in short axis cardiac MR images
Med. Image Anal.
(2011) - et al.
An improved level set method for incompressible two-phase flows
J. Comput. Fluids
(1998) - et al.
A coupled deformable model for tracking myocardial borders from real-time echocardiography using an incompressibility constraint
Med. Image Anal.
(2010)
Myocardial motion estimation from medical images using the monogenic signal
IEEE Trans. Image Process.
Automatic segmentation of echocardiographic sequences by active appearance motion models
IEEE Trans. Med. Imag.
Combined region and motion-based 3d tracking of rigid and articulated objects
IEEE Trans. Pattern Anal. Machine Intell.
The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods
IEEE Trans. Image Process.
Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart
Circulation
A methodology for evaluation of boundary detection algorithms on medical images
IEEE Trans. Med. Imag.
A multiple active contour model for cardiac boundary detection on echocardiographic sequences
IEEE Trans. Med. Imag.
Active contours without edges
IEEE Trans. Image Process.
Robust real-time myocardial border tracking for echocardiography: an information fusion approach
IEEE Trans. Med. Imag.
Dynamical statistical shape priors for level set-based tracking
IEEE Trans. Pattern Anal. Machine Intell.
Cited by (17)
Graph cut-based method for segmenting the left ventricle from MRI or echocardiographic images
2017, Computerized Medical Imaging and GraphicsCitation Excerpt :Another issue with active contours with echocardiographic images is that they can “bleed” in areas suffering from a signal dropout. To overcome this, authors often use shape priors (Dietenbeck et al., 2014, 2012; Pluempitiwiriyawej et al., 2005). While they do work, methods including a shape prior are prohibitively slow as well as mathematically and algorithmically complex.
A novel myocardium segmentation approach based on neutrosophic active contour model
2017, Computer Methods and Programs in BiomedicineCitation Excerpt :Computer aided diagnosis (CAD) system has been developed to identify the infarcted myocardium [9]. Many approaches have been proposed to identify and track myocardial borders [10], the ventricular cavity (i.e. endocardium) [11], epicardial boundary [12], and the full myocardium using echocardiographic images [13,14]. Ziwirn et al. [8] presented an endocardial boundary (inner boundary) detection method in myocardial images characterized by low signal-to-noise ratios.
Estimation of Large-Scale Organ Motion in B-Mode Ultrasound Image Sequences: A Survey
2015, Ultrasound in Medicine and BiologyCitation Excerpt :Common error measures are the mean squared error (MSE), the root mean square error (RMSE) and mean absolute difference (MAD). In the case of contours, the Hausdorff distance (Dietenbeck et al. 2014; Nascimento and Marques 2008) and the point-to-surface distance (Wang et al. 2010; Yang et al. 2008) have been used. Statistical tests include linear regression (Jacob et al. 2002; Li et al. 2013; Veronesi et al. 2006; Zahnd et al. 2013) and the Bland–Altman test (Duan et al. 2009; Jacob et al. 2002; Veronesi et al. 2006) and, for regions, Pearson's correlation coefficient test (Chalana et al. 1996; Mikic et al. 1998; Stoitsis et al. 2005).
Measurement of Myocardial Perfusion and Infarction Size Using Computer-Aided Diagnosis System for Myocardial Contrast Echocardiography
2015, Ultrasound in Medicine and BiologyAnalytic signal phase-based myocardial motion estimation in tagged MRI sequences by a bilinear model and motion compensation
2015, Medical Image AnalysisCitation Excerpt :In that work, a coarse-to-fine B-spline scheme allows for the effective and robust computation of the displacement, and also a pyramidal refinement scheme helps to deal with large motions. From the myocardial motion field, various further studies have been realized in order to extract motion-related information, such as myocardium segmentation (Dietenbeck et al., 2014), local deformation (Kar et al., 2014; Oubel et al., 2012), and local region tracking (Arif et al., 2014; Luo et al., 2014; Sun et al., 2011). For these information extraction methods, the key point is to estimate an accurate myocardium motion field.
SequenceMorph: A Unified Unsupervised Learning Framework for Motion Tracking on Cardiac Image Sequences
2023, IEEE Transactions on Pattern Analysis and Machine Intelligence