Active contours with neighborhood-extending and noise-smoothing gradient vector flow external force

Abstract

We propose a novel external force for active contours, which we call neighborhood-extending and noise-smoothing gradient vector flow (NNGVF). The proposed NNGVF snake expresses the gradient vector flow (GVF) as a convolution with a neighborhood-extending Laplacian operator augmented by a noise-smoothing mask. We find that the NNGVF snake provides better segmentation than the GVF snake in terms of noise resistance, weak edge preservation, and an enlarged capture range. The NNGVF snake accomplishes this with a reduced computational cost while maintaining other desirable properties of the GVF snake, such as initialization insensitivity and good convergences at concavities. We demonstrate the advantages of NNGVF on synthetic and real images.

1. Introduction

D uring the last two decades, variational and PDE-based methods for image segmentation and analysis have become standard tools [1]. Active contours or snakes which have deeply influenced variational approaches to image segmentation since their introduction [2] are curves that can conform to object boundaries or other image features under the influence of internal and external forces [2]. Generally, active contours can be categorized as parametric snakes [2] or as geometric snakes [3–5] according to their representation. Parametric snakes require an explicit representation while geometric snakes are defined implicitly. Here, we show how to construct an effective external force for parametric active contour models that can also be integrated into geometric active contours using a level set formulation [4].

Since the external force defines the evolution of an active contour, many external force models have been proposed [5–14]. Among these, the gradient vector flow (GVF) [10] has been most successful, as it provides a large capture range and the ability to capture concavities by diffusing the gradient vectors of an edge map generated from the image. Although the GVF model has proved effective and has widely been used in image segmentation, it has some disadvantages, such as a high computational cost, substantial noise sensitivity, and an inability to capture and preserve weak edges. Various improved models based on the GVF have been developed. For example, generalized gradient vector flow [11], harmonic gradient vector flow [12], motion gradient vector flow [13] and generalized dynamic directional gradient vector flow [14], but none of these are able to resolve all of the problems mentioned above.

We propose a novel external force for active models, called neighborhood-extending and noise-smoothing gradient vector flow (NNGVF), which incorporates a neighborhood-extended Laplacian operator mask and modifies the mask by adding a noise-smoothing mask. The proposed NNGVF snake outperforms the GVF snake in terms of computation, capture range, noise resistance, and weak edge preserving ability, while maintaining other desirable properties of the GVF snake such as initialization insensitivity and good convergence at concavities.

2. Background

2.1. Snakes: active contours

A snake is an elastic curve c(s) = [x(s),y(s)], s∊ [0, 1] which deforms to minimize the energy functional [2]:

$$E_{\text{snake}}=\int \frac{1}{2}\left(\alpha {\left|{\mathbf{c}}_{\text{s}}\right|}^2+\beta {\left|{\mathbf{c}}_{\text{ss}}\right|}^2\right)+E_{\text{ext}}\left[c\left(s\right)\right]ds,$$

(1)

where c_s(s) and c_ss(s) are the first and second derivative of c(s) with respect to s weighed by positive α and β, respectively. A typical external force for a gray-scale image I is E_ext = -|∇G _σ *I|, where G _σ is the Gaussian kernel with standard deviation σ and where * denotes convolution. Using standard variational methods, the Euler equation to minimize E_snake is expressed

$$\alpha {\mathbf{c}}_{\text{ss}}\left(s\right)-\beta {\mathbf{c}}_{\text{ssss}}\left(s\right)-\nabla E_{\text{ext}}=0.$$

(2)

This can be considered as a force balance equation:

$${\mathbf{F}}_{\text{int}}+{\mathbf{F}}_{\text{ext}}=0,$$

(3)

where F_int = αc_ss(s)-βc_ssss(s) and F_ext = -∇E_ext. The internal force F_int forces the snake contour to be smooth while the external force F_ext attracts the snake to the desired image features.

2.2. GVF: gradient vector flow external force

Typical shortcomings of the external force using the gradient vector of the edge map of the image include a limited capture range and convergence to concavities [10]. In order to solve these problems, Xu and Prince [10] proposed the GVF model. The GVF is a vector field v(x, y) = (u(x, y), v(x, y)) obtained by minimizing the following energy functional [15]:

$$E_{\text{GVF}}=\iint \mu \left(u_x^2+u_y^2+v_x^2+v_y^2\right)+{\left|\nabla f\right|}^2{\left|\mathbf{v}-\nabla f\right|}^{2}dxdy,$$

(4)

where f is the edge map of an image, and variable μ is a regularization parameter. The functions u(x, y) and v(x, y) are at least C² when E_GVF is minimized. The Euler equation to minimize E_GVF is:

$$u_t=\mu {\nabla }^{2}u-|\nabla f{|}^2\left(u-\nabla f\right)=0,$$

(5a)

$$v_t=\mu {\nabla }^{2}v-|\nabla f{|}^2\left(v-\nabla f\right)=0,$$

(5b)

where ∇² is the Laplacian operator.

3. NNGVF snakes

3.1. Extended neighborhood

A simple 4-neighborhood approximation to the Laplacian operator is:

$$\begin{array}{cc}\hfill {\nabla }^{2}f& =\frac{{\partial }^{2}f}{\partial x^2}+\frac{{\partial }^{2}f}{\partial y^2}=f\left(i,j+1\right)-2f\left(i,j\right)+f\left(i,j-1\right)+f\left(i+1,j\right)-2f\left(i,j\right)+f\left(i-1,j\right)\hfill \\ =f\left(i,j+1\right)+f\left(i,j-1\right)+f\left(i+1,j\right)+f\left(i-1,j\right)-4f\left(i,j\right)\hfill \end{array}$$

(6)

which can be achieved by convolution of the image with the mask:

$${\mathbf{G}}_{\mathbf{4}}=\left[\begin{array}{ccc}\hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 1\hfill & \hfill -4\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right].$$

(7)

There are a number of alternative Laplacian templates that may be used to extend the spatial extent of the Laplacian operator to a larger neighborhood, such as 8-neighborhood, 16-neighborhood, and 24-neighborhood approximations. By calculating the Laplacian operator over a larger neighborhood, more image information will be used. The 24-neighborhood mask that approximates the Laplacian operator is

$${\mathbf{G}}_{\mathbf{2}4}=\left[\begin{array}{ccccc}\hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill -24\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \end{array}\right].$$

(8)

3.2. Decomposition of the Laplacian operator

The particular Laplacian operator [16] can be expressed as the difference between an all-pass (AP) filter and a "round mean" (RM) filter, whose masks are shown in (10). The Laplacian operator can thus be defined as proportional to

$${\mathbf{G}}_{24}=\left(\mathbf{A}{\mathbf{P}}_{24}-\mathbf{R}{\mathbf{M}}_{24}\right),$$

(9)

$$\mathbf{A}{\mathbf{P}}_{\mathbf{2}4}=\left[\begin{array}{ccccc}\hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right],\mathbf{R}{\mathbf{M}}_{\mathbf{2}4}=\left[\begin{array}{ccccc}\hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill \\ \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill \\ \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 0\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill \\ \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill \\ \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill & \hfill 1/24\hfill \end{array}\right].$$

(10)

In this model, the AP filter is the 2D linear identity (do-nothing) filter, while the RM filter is a 2D low-pass filter. The difference yields high-frequency components over a large area. Since the purpose of the AP filter is to estimate the image at the center pixel, but is highly sensitive to noise [17], it is advisable to replace the AP filter with a better designed filter that can augment edge-preservation and noise robustness.

3.3. The proposed NNGVF external force

Motivated by Wang [17], we utilize the 5 × 5 noise smoothing (NS) filter shown in (12) instead of the AP filter, yielding considerably enhanced NS. We dub this model the NNGVF model. The NNGVF model is defined as the equilibrium solution of the following vector partial differential equation:

$$v_t=\mu \left(v*\mathbf{N}{\mathbf{S}}_{\mathbf{2}4}-v*\mathbf{R}{\mathbf{M}}_{\mathbf{2}4}\right)-{\left|\nabla f\right|}^2\left(v-\nabla f\right),$$

(11)

where μ is a regularization parameter and

$$\mathbf{N}{\mathbf{S}}_{\mathbf{2}4}=\left[\begin{array}{ccccc}\hfill 0\hfill & \hfill 0\hfill & \hfill 1/12\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1/12\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 1/12\hfill & \hfill 1/12\hfill & \hfill 1/3\hfill & \hfill 1/12\hfill & \hfill 1/12\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1/12\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1/12\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right].$$

(12)

In (11), NS₂₄ and RM₂₄ are 5 × 5 masks, which make use of larger areas of image information. Since the convolution is used in (11), the computational cost of NNGVF is greatly reduced relative to GVF.

4. Experimental results

Next, we demonstrate some desirable properties of the NNGVF snake and compare the performances of the NNGVF and GVF snakes. Since NNGVF is an improvement over GVF, we focus primarily on some common concerns encountered in snake-based image segmentation, which include (1) capture range enlargement and U-shape convergence, (2) weak edge preservation, (3) noise robustness, and (4) real images. The parameters for all snakes in our experiments are α = 0.1, β = 0 and time step τ = 1. The weight μ for the GVF and NNGVF snakes is set to 0.15 in all experiments unless otherwise stated.

4.1. Capture range enlargement and U-shape convergence

One of the advantages of the GVF snake is that its capture range is large. However, by expanding the calculations of the Laplacian operator to a larger neighborhood, more image information is used, so the NNGVF snake capture a larger range. Since the NNGVF snake computes the Laplacian operator using only a simple convolution, the computation time for the NNGVF is less than for GVF. As examples, we used the U-shape and room images, which have also been employed in [10, 11] to verify some general properties of NNGVF snake. Figure 1 shows that the NNGVF snake also can locate objects correctly from a distant initialization, can adhere to boundaries with gaps, and converges to concavity. For the room image, the execution time of NNGVF is 0.14 s while that of GVF is 0.42 s. For the U-shape image, the execution time of NNGVF is 0.17 s while that of GVF is 0.54 s. We can see that compared to the GVF snake, the NNGVF snake can capture a larger range with less computation time.

Figure 1

Results of snakes: (a) the GVF snake at iterations 60, (b) NNGVF snake at iterations 40, (c) GVF snake at iterations 90, and (d) NNGVF snake at iterations 50. In each panel, the left is the evolution of snake and the right is the force field.

4.2. Weak edge preserving

As pointed out in [18], imposing the Laplacian operator as a smoothing constraint fields has strong isotropic smoothing and poor edge preservation. Because of this, the GVF does not effectively preserve weak edges. Figure 2 shows the performance of the GVF and NNGVF snakes on a synthetic image, where there is a gap neighbored by a line segment. In Figure 2a, the GVF snake leaks and converges incorrectly to the line segment; while in Figure 2b, the NNGVF snake succeeds in preserving the weak edge.

Figure 2

Convergences of snakes: (a) the GVF snake, (b) NNGVF snake. In each panel, the left is the evolution of snake and right is the force field.

4.3. Noise robustness

Since the Laplacian operator used in GVF is sensitive to noise, the NNGVF also provides an advantage. The U-shape image used in [10] is employed here to generate noisy images. The first noisy image in Figure 3a is created by adding "salt & pepper" noise of intensity 0.06; the second one in Figure 3a by "speckle" noise with mean 0 and variance 0.08; the third one in Figure 3a by adding "Gaussian" noise of mean 0 and variance 0.01 and the fourth one in Figure 3a is by adding "Rayleigh" noise of mean 0 and variance 0.02. Figure 3 shows the results. It can be seen that the GVF snake was noticeably attracted to noise, but the proposed NNGVF snake moved into the concavity successfully.

Figure 3

Results of snakes: (a) the noisy images with top to bottom salt and pepper noise, speckle noise, Gaussian noise and Rayleigh noise, respectively; (b) convergences of GVF snakes, (c) convergences of NNGVF snakes.

4.4. Real images

Figure 4 shows segmentation results of the GVF and NNGVF snakes on two real medical images: one cardiac CT image (the first row) and one lung CT image (the second row). In these two examples, the algorithm must cope with noise and weak edges. For the cardiac image, we aim to extract the endocardium of the left ventricle, which is a weak edge. One can see that the GVF snake cannot convergence onto the desirable region on the image, whereas the NNGVF snake performs well on the weak edges and supplies good noise resistance. It can be seen that the GVF force field leaks at weak edges and gets trapped by the noise, while the NNGVF force field converges to the endocardium correctly. For the lung image, we aim to extract the parenchyma in the left and the cancer in the right part. The GVF snake again leaks at weak edges and gets trapped by the noise, whereas the NNGVF snake performs well on weak edges and supplies good noise resistance. Furthermore, in Figure 4b, d, the execution times of NNGVF are 1.26 and 1.15 s, respectively; while in Figure 4a, c, the execution times of GVF are 3.92 and 3.78 s, respectively.

Figure 4

Convergences of snakes: (a) the GVF snake at iterations 250, (b) NNGVF snake at iterations 120, (c) GVF snake at iterations 200, and (d) the NNGVF snake at iterations 100. In each panel, the left is the evolution of snake and the right is the force field.

5. Conclusion

We proposed a novel external force called NNGVF for active contours. The NNGVF snake deploys the GVF as a convolution operation using a neighborhood-extending Laplacian mask, modifying the mask to improve noise-smoothing, yields a good performance in terms of capture range, weak edge preservation, and noise robustness while maintaining the other desirable properties of GVF, such as initialization insensitivity and good convergence at concavities. The experimental results showed that the NNGVF snake outperforms the GVF snake in terms of computation as well.