A new method for linear feature and junction enhancement in 2D images based on morphological operation, oriented anisotropic Gaussian function and Hessian information
Introduction
Many objects such as retinal vessels, neurites and plant roots have linear features. Information about these structures can be used in biomedical and other types of applications. By analyzing vessel structures, diseases that involve structural or functional changes can be investigated [1], [2]. For example, hypertension may result in focal constriction of retinal arteries and arteriosclerosis can cause the arteries to show a copper or silver color [3], [4]. Moreover, the number and morphology of neuron cells can be used to predict or diagnose brain diseases such as Alzheimer׳s disease, which is characterized by the loss of neurons and synapses in the cerebral cortex and certain subcortical regions [5]. Therefore, the detection, segmentation and tracking for linear structures in images can release the biologists from heavy burden of manual work and improve efficiency.
However, the images that are to be segmented are not always of high quality due to distortions during acquisition, processing, compression, storage, transmission and reproduction, which will affect the result of segmentation or tracking [6]. The enhancement of linear features can make the image processing steps easier and greatly improve the result of feature segmentation. In medical image analysis, for example, enhancement of vessels can improve the visualization of vessels and small vessel delineation. It can also provide the input for vessel segmentation, centerline extraction and tracking.
In this paper, novel algorithms are proposed to enhance linear features as well as enhance junction regions without any suppression. Firstly, we propose an adaptive multi-scale morpho-Gaussian filter to enhance, reconnect and smooth linear features. From the Hessian matrix, the orientation of the features and a measurement based on the eigenvalues are obtained. At each pixel along its orientation, we carry out a linear morphological operation so that linear structures are enhanced and noisy structures are suppressed. The intensity profile of the linear structure׳s cross section can be approximated by a Gaussian curve [7], [8]. Multi-scale locally oriented anisotropic Gaussian templates are applied in each morphologically enhanced window to smooth the linear structures in our work. To match linear structures with various widths, the Gaussian function is used in multiple scales. The enhanced and reconnected features using the morphological operation are smoothed by using this multi-scale-anisotropic Gaussian function. Then a multi-scale Hessian-based filter is used to further enhance the linear structures. The enhancement results of existing methods in junction regions are not satisfactory because junctions are the intersection of several linear structures and have multiple orientations. To solve this issue, a novel method for junction enhancement is proposed. We first decompose the junctions into separate linear structures or branches, and then we enhance the linear features along each branch. In Section 2, we give a brief review of some of the existing approaches. Section 3 gives a brief overview about our algorithm. In Section 4, the linear feature enhancement approach is described. Our junction enhancement method is described in Section 5. We give experimental results and statistical analysis in Section 6. The paper is concluded in Section 7.
Section snippets
Related work
Due to the importance of linear feature enhancement, a variety of enhancement methods have been developed recently. Hessian based approach is one of the most popular methods, because the second order structure in the Hessian matrix describes the intensity variations for each point [9]. In Sato et al. and Lorenz et al.׳s work, eigenvalues of the Hessian matrix were analyzed to determine the local likelihood of vesselness [10], [11]. They constructed a line-structure-ness measure using the
Overview of the proposed method
We would like to give an overview of the proposed method. The algorithm steps for the whole procedure are:
- 1.
Calculate the orientation and the intensity variation for each pixel based on Hessian information.
- 2.
Correct the orientation and fill in the dark shadows next to the linear structures.
- 3.
Apply the adaptive multi-scale morpho-Gaussian filter:
- (a)
In a rectangular window, enhance each linear feature and suppress noisy features.
- (b)
Use the multi-scale anisotropic Gaussian kernel to smooth linear features.
- (a)
- 4.
Hessian measurements and preprocessing
Hessian matrix involves the second order gradient of images. Analysis of the eigenvalues of a Hessian matrix can provide a measurement for the principal direction and the vesselness of a linear structure. Assuming the two eigenvalues of the Hessian matrix of a 2D image are λ1 and λ2, and they satisfy (assuming bright structures). The eigenvalues represent the variation of the intensity in a linear structure׳s principal orientation and its perpendicular orientation. So for a linear
Junction enhancement
Junctions are the regions where three or more linear structures meet. We call these linear structures at junction regions ‘branches’. Hessian-based methods such as Frangi et al.׳s method [12] have a problem of junction suppression [17]. In the proposed method, we do not have the junction suppression problem in the morpho-Gaussian filter step as our algorithm only uses the orientation information and the intensity variation from the Hessian matrix. Besides, enhancement of linear features in the
Experimental results and comparisons
We evaluate the performance of the proposed method through applications to 49 images which include synthetic images, neurite images, leaf vein images and retinal images. We compare our results with four other methods: Tankyevych et al.׳s method [24], Kroon et al.׳s method [22], Frangi et al.׳s method [12] and Shikata et al.׳s method [15]. The reason we choose these four methods is that Frangi et al.׳s method and Shikata et al.׳s method are two classic Hessian methods for linear feature
Conclusions
In this paper, we propose a new method to enhance linear features and junctions in images. We enhance the linear features and junctions in two steps. In the linear structure enhancement, a multi-scale morpho-Gaussian filter is proposed to enhance the linear features. This method uses the enhancement function of the morphological operation and the smoothing function of the Gaussian filter. The use of multiple scales approach ensures that the linear structures with various widths can all be
Conflict of Interest
None declared.
Acknowledgments
We would like to acknowledge and extend our gratitude to Xiao Tan at the University of New South Wales, Canberra, Australia, for his comments on this paper. We thank the anonymous reviewers for their helpful and constructive comments on this paper. The preliminary conference version of this paper was presented in [45].
Ran Su obtained her PhD degree from the University of New South Wales, and spent her research co-training at CSIRO. Her main research interest is image analysis, pattern recognition and machine learning.
References (45)
- M.J. Cree, D. Cornforth, H.F. Jelinek, Vessel segmentation and tracking using a two-dimensional model, in: Proceedings...
- Y. Kawata, N. Niki, T. Kumazaki, An approach for detecting blood vessel diseases from cone-beam CT image, in:...
- O. Chutatape, L. Zheng, S.M. Krishnan, Retinal blood vessel detection and tracking by matched Gaussian and Kalman...
- et al.
Measurement of retinal vessel widths from fundus images based on 2-D modeling
IEEE Trans. Med. Imaging
(2004) Neuropathologic changes in Alzheimer׳s disease
J. Clin. Psychiatr.
(2003)- et al.
Image quality assessmentfrom error visibility to structural similarity
IEEE Trans. Image Process.
(2004) - et al.
Detection of blood-vessels in retinal images using two-dimensional matched-filters
IEEE Trans. Med. Imaging
(1989) - H. Li, W. Hsu, M.L. Lee, H. Wang, A piecewise Gaussian model for profiling and differentiating retinal vessels, in:...
- Y. Chen, L. Wang, L. Shi, D. Wang, P.-A. Heng, T.-T. Wong, X. Li, Structure-preserving multiscale vessel enhancing...
- C. Lorenz, I.C. Carlsen, T.M. Buzug, C. Fassnacht, J. Weese, Multi-scale line segmentation with automatic estimation of...
Efficient computation of Hessian-based enhancement filters for tubular structures in 3D images
IRBM
Vessel enhancement filter using directional filter bank
Comput. Vis. Image Underst.
Detection and enhancement of line structures in an image by anisotropic diffusion
Pattern Recognit. Lett.
Vesselness enhancement diffusion
Pattern Recognit. Lett.
Flux-based anisotropic diffusion applied to enhancement of 3-D angiogram
IEEE Trans. Med. Imaging
Vessel enhancing diffusiona scale space representation of vessel structures
Med. Image Anal.
Cited by (43)
A GAN-based deep enhancer for quality enhancement of retinal images photographed by a handheld fundus camera
2022, Advances in Ophthalmology Practice and ResearchMC-DMD: A data-driven method for blood vessel enhancement in retinal images using morphological closing and dynamic mode decomposition
2022, Journal of King Saud University - Computer and Information SciencesCitation Excerpt :A second order elliptical model was used by taking all the Eigen values of a Hessian matrix. Su et al. (2014) proposed an approach based on Gaussian filter which enhances and smoothens linear features, removing noise, and solves junction suppression problem seen in numerous existing methods. Different from vesselness, Meijering et al. (2004) introduced an enhancement method to measure the neuriteness.
Vessel enhancement using Multi-scale Space-Intensity domain Fusion Adaptive filtering
2021, Biomedical Signal Processing and ControlThe multiscale bowler-hat transform for blood vessel enhancement in retinal images
2019, Pattern RecognitionCitation Excerpt :A wide range of vessel enhancement methods have been proposed (see [2] and [1] for a recent review). These include Hessian [3–5], Phase Congruency Tensor [6,7], mathematical morphology [5,8,9], adaptive histogram equalisation [10] based approaches and many others [11–18]. However, many of these methods still have considerable issues when faced with variations in contrast, high levels of noise, variation in image features (e.g. lines vs junctions; retention of network connectivity), and complexity of method parameter space.
A novel active contour model for medical images via the Hessian matrix and eigenvalues
2018, Computers and Mathematics with ApplicationsConnotations of pixel-based scale effect in remote sensing and the modified fractal-based analysis method
2017, Computers and GeosciencesCitation Excerpt :Strahler et al. (1986) built a framework for distinguishing between the scene, which is real and exists on the ground; Woodcock and Strahler (1987) proposed the idea for selecting the optimal spatial resolution by using average Local Variance; Cracknell (1998) discussed the question of pixel size or instantaneous field-of-view (IFOV) both from the simple geometrical point of view and from a more physical point of view. Ming et al., (2008, 2011, 2013) used the Modified Average Local Variance (MALV) method based on constant ground area (variable window size and variable resolution) to study the optimal spatial resolution of different features on remote sensing images; The above studies are almost based on geo-statistics methods and utilize geometry to analyze the linear features (Su et al., 2014) of remote sensing images. However, both linear and non-linear features (Shokoohi et al., 2013) are prominent in remote sensing images.
Ran Su obtained her PhD degree from the University of New South Wales, and spent her research co-training at CSIRO. Her main research interest is image analysis, pattern recognition and machine learning.
Changming Sun received the PhD degree in the area of computer vision from Imperial College London in 1992. Then, he joined CSIRO Computational Informatics, Australia, where he is currently a principal research scientist carrying out research and working on applied projects. His research interests include computer vision and photogrammetry, image analysis, and pattern recognition. He has served on the program/organizing committees of various international conferences. He is an Associate Editor for EURASIP Journal on Image and Video Processing, a SpringerOne journal. Dr. Sun is a member of the Australian Pattern Recognition Society.
Chao Zhang obtained his PhD degree from the University of New South Wales, and spent his research co-training at CSIRO. His main research interest is image analysis and pattern recognition.
Tuan D. Phamreceived his PhD in 1995 from the University of New South Wales (Sydney, Australia). His current research interests focus on image analysis and pattern recognition in medicine and biology. His research has been funded by government agencies, academic institutions, and industry. Dr. Pham is a senior member of the Institute of Electrical and Electronics Engineers (IEEE), editorial board member and associate editor of several journals and book series. He has served as chair and technical member of a number of international conferences, and the founder of the International Symposium on Computational Models for Life Sciences (CMLS) and The International Aizu Conference on Biomedical Informatics and Technology (ACBIT).