An iterative possibilistic knowledge diffusion approach for blind medical image segmentation

doi:10.1016/j.patcog.2018.01.024

Pattern Recognition

Volume 78, June 2018, Pages 182-197

https://doi.org/10.1016/j.patcog.2018.01.024 Get rights and content

Highlights

•
A novel region-growing segmentation method based on possibilistic theory is proposed.
•
Region-growing process is iteratively performed at the possibilistic knowledge representation level.
•
Possibility theory allows adequate semantic knowledge modeling without huge constraints.
•
Validation is done in the context of pixel classification using both real and synthetic data.
•
Proposed approach shows remarkable stable behaviour during quantitative assessment.

Abstract

This paper presents an image segmentation method imitating human focusing visual attention in image interpretation using possibilistic knowledge modeling concepts. The proposed pixel level method consists on the Iterative Possibilistic Knowledge Diffusion (IPKD) on immediate neighbourhood pixels. The advantage of this mechanism is to provide iterative diffusion of per-pixel certain knowledge to surrounding pixels in order to progressively refine the segmentation process. The diffusion process is achieved using image smoothing techniques such as Nagao and Gabor filtering, mean filtering and anisotropic diffusion. Those diffusion techniques are then compared in the possibilistic knowledge representation space. The merit of a possibilistic knowledge representation, rather than a grey-level sensor based representation, is demonstrated by both experimental and synthetic data. Producing the lowest error rates, possibilistic knowledge diffusion using Nagao filter is adopted for the approach assessment. Experimental results using synthetic images as well as mammographic images from MIAS (Mammographic Image Analysis Society) data-base, are performed in order to assess the efficiency of the proposed segmentation method according to the visual criterion as well as some quantitative criteria. IPKD's performance (in terms of recognition rate, 94.37% and global predictive rate, 92.18%) is compared with three relevant reference methods: level-set, Fuzzy C-Mean and region growing methods. The IPKD approach outperforms the other three methods, respectively, at the recognition rates of 89.77%, 84.43% and 88.11% and at the global predictive rates of 87.86%, 89.72% and 84.04%. Noise-sensitivity experiments have been conducted on synthetic as well as on real images. The proposed IPKD approach outperforms the three reference methods and in addition, exhibits a desired stability behaviour.

Graphical abstract

Introduction

In the field of image analysis and interpretation, various kinds of information imperfections are present in all image-processing phases (i.e. preprocessing, feature extraction, image analysis and scene interpretation). Epistemic, due to system constructs, and random or aleatory, inherent to physical observation processes, are generally the terms used to categorize information-based uncertainties. A recent pertinent discussion about uncertainty and its typology (ambiguity, imprecision, vagueness, incompleteness, …, etc.) is presented in chapter 3 of [1]. That discussion is important since a better understanding of information imperfections at each level of a system allows better modelling and consequently better system design to cope with the consequences of those imperfections on systems that support image exploitation and interpretation.

This paper considers a new approach for image segmentation based on possibilistic concepts. The aim of a segmentation process is to distinguish homogeneous regions within an image that belong to different objects. For instance, the segmentation process can be based on finding the maximum homogeneity in grey levels within the identified regions, i.e. the basic information delivered at the output of an imaging sensor. Regions resulting from a segmentation process have to display a uniform behaviour with regard to a considered set of features. That is challenging. First, when can we consider that a behaviour is uniform with respect to a feature or a set of features? Second, given that objects have uniform boundaries, then we are faced with the boundary location problem. In fact, for marking the region boundaries and for locating uniform ones, the uniformity degree of neighbouring pixels shall be considered.

Moreover, several challenges related to image segmentation require attention. One important challenge is related to the selection of the suitable approach to isolate different objects from their background. Another important challenge involves measuring the performance of a segmentation approach and assessing its impact on the global image and scene interpretation.

Image segmentation issues have been approached from a wide variety of perspectives, and different techniques have been developed are to perform that task. These techniques are generally grouped into four major categories: a) clustering-based and histogram thresholding approaches; b) edge-based approaches; c) region-growing based approaches, and finally; d) hybrid approaches combining both region-growing and edge-based techniques. The four image segmentation approaches (a,b,c,d) implement different “understandings” of the human-based image segmentation process. Nevertheless, in all cases, the major encountered difficulty is related to the fact that all image segmentation approaches mainly depend on the “nature” of the spatial knowledge to be used and its spatial diffusion as well as the set of similarity criteria which are strongly based on the physical parameters measured by the imaging sensor in action: consequently, other sources of knowledge are not being integrated.

The focus of the present paper is to offer a region-growing image segmentation approach (category c above) based upon the use of possibility theory concepts that imitates the reasoning scheme of a human when analysing an observed image in terms of its constituting homogeneous regions. Based on the observed features of grey level intensity, colour, texture, patterns, shapes, etc., a human understand image by first, segmenting the whole image into regions/objects. This initial segmentation task is realized by “visually” locating sub-regions, called “reference seed”. Reference seeds are homogenous small regions having semantic visual significance or satisfying some simple similarity criteria: feature-based homogeneity, membership to a known structure or a thematic class. Then, a second “iterative” step follows. It consists on spreading the semantic homogeneous seeds into their spatial context using a set of similarity criteria. Additional sources of knowledge are then used to enlarge the initial homogenous regions. This process leads to a global segmented image over which other interpretation tasks may be conducted.

In this paper, an iterative region growing image segmentation approach based on the use of possibility theory is proposed and evaluated. The main reason behind the use of possibility theory is that this theory allows adequate semantic knowledge modeling without huge constraints. Possibilistic concepts offer simple means for modelling human reasoning related to spatial similarity and to contextual knowledge diffusion. The proposed possibilistic region growing image segmentation approach consists on the possibilistic modeling of different semantic classes present in the observed scene. The observed image is then projected into the classes’ possibilistic representation space forming, thus, possibilistic class maps. Finally, the region-growing process is conducted at the possibility map level.

The paper is organized as follows. Section 2 is devoted to review basic concepts of knowledge diffusion for region- based approaches in image segmentation. A brief introduction to possibility theory is presented in Section 3. The proposed possibilistic knowledge approach is then detailed in Section 4. Section 5 presents empirical results to assess the impact of possibilistic spatial knowledge diffusion allowing the evaluation of the proposed approach. Section 6 is the conclusion.

Section snippets

Related work for knowledge diffusion and region based approaches in image segmentation

An image represents a partial view of an observed scene obtained by interpreting physical measures from sensors. Region-growing based image segmentation methods aim at the identification of the “constituting elements” contained within the image by grouping image pixels into homogeneous regions formed by linked pixels. Each region is assumed to meet an homogeneity criterion according to a common property: its constituting pixels belong to a same semantic entity or a same thematic class. The

A very brief introduction to possibility theory

A very brief introduction to possibility theory is given here since a sufficient description, pertinent to the discussion here, has already been presented in [2], [3], [7]. If the available knowledge is ambiguous and encoded by a membership function, i.e. a fuzzy set, defined over the decisions set, $Ω = {C_{1}, C_{2}, \dots, C_{M}}$ , the possibility theory transforms the membership function into a possibility distribution π. Then the realization of each event A is bounded by a possibilistic interval defined by a

The proposed IPKD approach

This paper proposes an image segmentation approach that exploits to a larger extent the spatial information contained within an image. Section V presents empirical results in order to qualify and quantify that extent. The proposed approach, inspired from region-growing methods, consists, first, in representing available diverse knowledge sources in the possibility theory formalism. The second step is to apply a possibilistic knowledge diffusion process based on contextual information in a

Empirical results

This section presents a quantitative and qualitative evaluation of the possibilistic knowledge diffusion approach using two sources of data: a synthetic image and a set of mammographic medical images from MIAS data base “Mammographic Image Analysis Society” [51].

The synthetic test image (illustrated in Fig. 3) is composed of a basic miniature image containing four circular disks (class C₁) and the image background (class C₂). The use of several sizes constitutes a first indicator allowing to

Conclusions

In this paper, an iterative possibilistic knowledge diffusion (IPKD) approach for image segmentation has been proposed. A priori expert knowledge used by the proposed approach is limited to the set of thematic classes assumed to be present in the analysed image as well as a seed representative zone for each considered class. The Nagao-based possibilistic knowledge diffusion is retained as the method producing the best segmentation results. In addition, this method has the important property of

Acknowledgments

Authors acknowledge the technical assistance of Dr. Héla Fourati Mseddi, working in Medical Imagery Service of CHU Hédi Chaker Sfax, (Tunisia), for providing ground truth of mammographic images used in the present work.

Imene Khanfir Kallel, Ph.D., Electrical engineering and Ph.D. (ENIS-Tunisa, 2002 and 2010). Actually associate-professor in ISBS, member of CEM-Lab (Sfax-Tunisia) and associate-researcher in iTi-Dept (IMT-Atlantique, Bretagne-France). She's editorial board member of i-manager's (JDP) since 2015. She works on knowledge processing, analysis and interpretation and has published more than 30 papers in reputed journals and conferences.

References (59)

B. Alsahwa et al.
A method of spatial unmixing based on possibilistic similarity in soft pattern classification
Fuzzy Inf. Eng.
(2016)
FanJ. et al.
Seeded region growing: an extensive and comparative study
Pattern Recognit. Lett.
(2005)
S. Ammar Bouhamed et al.
MasmoudiB. Solaiman Feature selection in possibilistic modeling
Pattern Recognit.
(2015)
YinP.-Y. et al.
A fast iterative scheme for multilevel thresholding methods
Signal Process.
(1997)
WangL. et al.
Simultaneous segmentation and bias field estimation using local fitted images
Pattern Recognit.
(2018)
S. Ruan et al.
Fuzzy Markovian segmentation in application of magnetic resonance images
Comput. Vision Image Understand.
(2002)
DouW. et al.
A framework of fuzzy information fusion for the segmentation of brain tumor tissues on MR images
Image Vis. Comput.
(2007)
C.D. Kermad et al.
Automatic image segmentation system through iterative edge–region co-operation
Image Vision Comput.
(2002)
M. Nagao et al.
Edge preserving smoothing
Comput. Graph. Image Process.
(1979)
L. Zadeh
Fuzzy Sets as the Basis for a Theory of Possibility
Fuzzy Sets Syst.
(1978)

KimI.Y. et al.

Efficient image labeling based on Markov random field and error backpropagation network

Pattern Recognit.

(1993)

J. Johansson et al.

An approach for modelling interdependent infrastructures in the context of vulnerability analysis

Reliability Engineering & System Safety

(2010)

ChengH. et al.

Approaches for automated detection and classification of masses in mammograms

Pattern Recognit.

(2006)

É. Bossé et al.

Fusion of Information and Analytics for Big Data and IoT

(2016)

B. Alsahwa et al.

Iterative refinement of possibility distributions by learning for pixel-based classification

IEEE Trans. Image Process.

(2016)

A. Mencattini et al.

Breast mass segmentation in mammographic images by an effective region growing algorithm

Adv. Concepts Intel. Vis. Syst.

(2008)

M.S. Mouchaweh

Semi-supervised classification method for dynamic applications

Fuzzy Sets Syst.

(2010)

R.C. Gonzalez et al.

Digital Image Processing

(2002)

S.R.R. Dhara et al.

Determination of breast cancer area from mammography images using thresholding method

Int. J. Innov. Res. Adv. Eng.

(2017)

M. Adel et al.

Statistical segmentation of regions of interest on a mammographic image

EURASIP J. Adv. Signal Process.

(2007)

A. Zehtabian et al.

An adaptive framework for spectral-spatial classification based on a combination of pixel-based and object-based scenarios

Earth Sci. Inform.

(2017)

S. Geman et al.

Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images

IEEE Trans. Pattern Anal. Mach. Intell.

(1984)

G. Loum et al.

Caractérisation de textures à l'aide d'un codage directionnel local

Afrique Science: Revue Internationale des Sciences et Technologie

(2016)

M.A. Aguilar et al.

Classification of urban areas from GeoEye-1 imagery through texture features based on Histograms of Equivalent Patterns

Eur. J. Remote Sens.

(2016)

M. Bouhlel et al.

Melanoma-Pattern Extraction using Histogram-thresholding Approach

Off. J. Egypt. Soc. Med. Educ. Sci. Med. J. Sci. Med J.(ESCME)

(2002)

O.J. Tobias et al.

Image segmentation by histogram thresholding using fuzzy sets

IEEE Trans. Image Process.

(2002)

ChanT.F. et al.

Active contours without edges

IEEE Trans. Image Process.

(2001)

K. Nawres et al.

Segmentation d'images par contours actifs: application à la détection du ventricule gauche dans les images de scintigraphie cardiaque

SETIT

(2005)

ZhaoW. et al.

Active contour model based on local and global Gaussian fitting energy for medical image segmentation

Optik - Int. J. Light Electron Opt.

(2018)

Cited by (19)

2D Image head pose estimation via latent space regression under occlusion settings
2023, Pattern Recognition
Citation Excerpt :
In this section, we introduce a procedure capable of generating synthetic occlusions in images, and describe the datasets used for the head pose estimation. The generation of synthetic data for training in deep learning frameworks has become ever more common and has been proven to be essential in the enlargement of training sets and in improving the generalization and accuracy of learning models e.g., medical segmentation [41], autonomous driving [42] and pose estimation [20,43]. We use existing 2D image head pose datasets that contain thousands of images and respective ground truth pose annotations.
Head orientation is a challenging Computer Vision problem that has been extensively researched having a wide variety of applications. However, current state-of-the-art systems still underperform in the presence of occlusions and are unreliable for many task applications in such scenarios. This work proposes a novel deep learning approach for the problem of head pose estimation under occlusions. The strategy is based on latent space regression as a fundamental key to better structure the problem for occluded scenarios. Our model surpasses several state-of-the-art methodologies for occluded HPE, and achieves similar accuracy for non-occluded scenarios. We demonstrate the usefulness of the proposed approach with: (i) two synthetically occluded versions of the BIWI and AFLW2000 datasets, (ii) real-life occlusions of the Pandora dataset, and (iii) a real-life application to human-robot interaction scenarios where face occlusions often occur. Specifically, the autonomous feeding from a robotic arm.
Fighting against terrorism: A real-time CCTV autonomous weapons detection based on improved YOLO v4
2022, Digital Signal Processing: A Review Journal
Citation Excerpt :
Expanding a model's training data with synthetic datasets is a reliable data augmentation strategy that can effectively enhance model performance. In the field of computer vision, such as license plate recognition [22] and medical image segmentation and detection [23], the scheme has been widely employed for deep network training. Hattori H et al. [24] proposed that synthetic data can compensate for the absence of real training data.
In recent years, deep learning has demonstrated tremendous potential in the real world. Object detection is a critical real-world task for deep learning. You Only Look Once (YOLO) object detection model recognizes interesting regions in images with impressive accuracy and real-time performance. The objective of this paper is to apply object detection to the field of security and counter-terrorism. Individuals are protected from violence by recognizing and locating the guns on closed-circuit television (CCTV). This paper presents a real-time detection approach for CCTV autonomous weapons based on YOLO v4. For the characteristics of CCTV scenarios, we propose the YOLO v4 backbone with Spatial Cross Stage Partial-ResNet (SCSP-ResNet). Meanwhile, the receptive field enhancement module is shown to capture fine semantic features of high-dimensional small objects. The Fusion-PANet (F-PaNet) module has been used to fuse multi-scale information to improve the model's perceptive power on the region of interest. Furthermore, we merge synthetic and real-world datasets to comprehensively investigate the effects of synthetic datasets on detectors. The experimental results reveal that our suggested detection model improves mAP (mean Accuracy Precision) and inference time by 7.37% and 4.2%, respectively. The model's parameter is reduced by 0.349 BFLOP/s(billion floating point operations per second). The proposed detector outperforms the baseline model in terms of accuracy, real-time, and robustness.
An intelligent quality-based approach to fusing multi-source possibilistic information
2020, Information Fusion
Citation Excerpt :
The recognition rates, illustrated in Table 16 for the impoverished datasets, show the advantage of the possibility theory in data modeling compared to the probability theory and the SVM classifier in poor data environments (information incompleteness). Same kind of advantages have been confirmed from results that have been obtained for different kind of applications namely in pattern recognition and image segmentation in the processing of poor-quality images [32,34-37]. It is worth mentioning that the size of databases in Table 16 does not exceed 800 samples.
Recently, Yager and Petry were proposing a quality-based methodology to combine data provided by multiple probabilistic sources to improve the quality of information for decision-makers. This paper offers a sort of companion paper that adapts this methodology to possibilistic sources. Possibility theory is particularly well suited to cope with incomplete information from poor-data sources. The methodology and algorithms used for the probabilistic approach are adapted for the possibilistic case. Both approaches are then compared by the means of a numerical example and four experimental benchmark datasets: one, the IRIS data set, being data-poorer than the three other ones (Diabetes dataset, Glass dataset and Liver-disorder dataset). A vector representation is introduced for a possibility distribution as in the probabilistic case and, the Gini's formulation of entropy is being used. However, the Gini's entropy has to be used differently than with the probabilistic case. This has an impact on the selection of subsets. A fusion scheme is designed to select the best-quality subsets according to two information quality factors: quantity of information and source credibility. Results obtained from comparison of both approaches on the four experimental benchmarks confirm the superiority of the possibilistic approach in the presence of information scarcity or incompleteness.
A deep Coarse-to-Fine network for head pose estimation from synthetic data
2019, Pattern Recognition
Citation Excerpt :
In the past decades, researchers have made impressive progress on 3D object modeling and synthesis. Synthesized data has been applied for deep network training in many computer graphics and vision tasks, e.g., autonomous driving [27], license plate recognition [28], 3D reconstruction [29], scene understanding [30], human detection and pose estimation [31], and medical image segmentation [32] and detection [33], which have proved that synthetic data can help to achieve good performance. In the follow-up phase of the study, we find that the head pose datasets (e.g,.
Various applications of human-computer interaction are based on the estimation of head pose, which is challenging due to different facial appearance, inhomogeneous illumination, partial occlusion, etc. In this paper, we propose a deep neural network following the Coarse-to-Fine strategy to estimate head poses. The scheme includes two branches: Coarse classification phase classifying the input image into four categories, and Fine Regression phase estimating the accurate pose parameters. The two sub-networks are trained jointly. To tackle the problem of insufficient annotated data in training process, we design a rendering pipeline to synthesize realistic head images and generate an annotated dataset with a collection of 310k head poses. The results on benchmark datasets and synthetic dataset validate the effectiveness of our approach, as well as the results on images with diverse illumination, occlusion, and motion blur. Moreover, our method can be easily extended to estimate head poses on depth images.
A local mean and variance active contour model for biomedical image segmentation
2019, Journal of Computational Science
Citation Excerpt :
Lei et al presented a local hybrid image fitting (LHIF) model by constructing a local hybrid image fitting energy based on two different local fitted images [47]. Some similar methods were recently proposed in [48,49] which can deal with intensity inhomogeneity as the LBF model and the LHIF model. However, to some extent these models are still sensitive to initial contour, which limits their practical applications.
To effectively segment the biomedical images with intensity inhomogeneity, a local mean and variance (LMV) active contour model is presented in this paper. In the model, the distribution of intensity belonging to each region is assumed as a Gaussian distribution with spatially varying mean and variance. At the same time, an energy function is defined and incorporated into a variational level set formulation. The curve evolution equation is then obtained from energy minimization. Because both image local mean and variance are considered, the proposed model can more effectively deal with the images with intensity inhomogeneity than the existing active contour model. Experimental results on synthetic and real images show the advantages of our method in terms of both effectiveness and robustness. Compared with the local binary fitting (LBF) model and the local hybrid image fitting (LHIF) model, our model is much most computationally efficient and much least sensitive to the initial contour.
Automated pulmonary nodule detection in CT images using deep convolutional neural networks
2019, Pattern Recognition
Citation Excerpt :
It is also a challenging task because there are many pulmonary nodules with various sizes, shapes, locations and types which are shown in Fig. 1. Recently, the revolution of deep learning has attracted many researchers to pay their attention to the applications of deep learning in CAD with the extraordinary learning power [6–9]. Current automated pulmonary nodule detection systems mainly consist of two stages: (1) nodule candidate detection; (2) false positive reduction [10–15].
Lung cancer is one of the leading causes of cancer-related death worldwide. Early diagnosis can effectively reduce the mortality, and computer-aided diagnosis (CAD) as an important way to assist doctors has developed rapidly. In particular, automated pulmonary nodule detection in computed tomography (CT) images is crucial to CAD. It is a challenging task to quickly locate the exact positions of lung nodules. In this paper, a novel automated pulmonary nodule detection framework with 2D convolutional neural network (CNN) is proposed to assist the CT reading process. Firstly, we adjust the structure of Faster R-CNN with two region proposal networks and a deconvolutional layer to detect nodule candidates, and then three models are trained for three kinds of slices for later result fusion. Secondly, a boosting architecture based on 2D CNN is designed for false positive reduction, which is a classifier to distinguish true nodules from the candidates. The misclassified samples are still kept for retraining a model which boosts the sensitivity for pulmonary nodule detection. Finally, the results of these networks are fused to vote out the final classification results. Extensive experiments are conducted on LUNA16, and the sensitivity of nodule candidate detection achieves 86.42%. For the false positive reduction, the sensitivity reaches 73.4% and 74.4% at 1/8 and 1/4 FPs/scan, respectively. It illustrates that the proposed method can obviously achieve accurate pulmonary nodule detection.

View all citing articles on Scopus

Basel Solaiman, Ph.D., Telecommunication engineering (ENST, 83), Ph.D. and HDR (Université de Rennes-I, 88 and 97). Actually, he is professor and head of iTi-department in IMT Atlantique, Bretagne. He has published several academic books, book chapters and over 200 journal papers. Application domains of his research activities: medical, remote sensing, underwater imaging and knowledge mining.

Éloi Bossé, Ph.D., (M’11), received the B.A.Sc. (79), M.Sc. (81) and Ph.D (90) degrees from Université Laval, QC, in Electrical Engineering. He worked on signal processing, high resolution spectral analysis, information fusion and decision support at Communications Research Centre, Defence Research Establishments at Ottawa and Quebec. He has published over 200 papers in journals, book chapters, conference proceedings and technical reports. Dr. Bossé is actually a researcher and adjunct professor at McMaster University, Hamilton Canada as well as an associate researcher at IMT-Atlantique, Brest, France.

View full text

An iterative possibilistic knowledge diffusion approach for blind medical image segmentation

Highlights

Abstract

Graphical abstract

Introduction

Section snippets

Related work for knowledge diffusion and region based approaches in image segmentation

A very brief introduction to possibility theory

The proposed IPKD approach

Empirical results

Conclusions

Acknowledgments

Fuzzy Inf. Eng.

Pattern Recognit. Lett.

Pattern Recognit.

Signal Process.

Pattern Recognit.

Comput. Vision Image Understand.

Image Vis. Comput.

Image Vision Comput.

Comput. Graph. Image Process.

Fuzzy Sets Syst.

Pattern Recognit.

Reliability Engineering & System Safety

Pattern Recognit.

Fusion of Information and Analytics for Big Data and IoT

Iterative refinement of possibility distributions by learning for pixel-based classification

IEEE Trans. Image Process.

Breast mass segmentation in mammographic images by an effective region growing algorithm

Adv. Concepts Intel. Vis. Syst.

Semi-supervised classification method for dynamic applications

Fuzzy Sets Syst.

Digital Image Processing

Determination of breast cancer area from mammography images using thresholding method

Int. J. Innov. Res. Adv. Eng.

Statistical segmentation of regions of interest on a mammographic image

EURASIP J. Adv. Signal Process.

An adaptive framework for spectral-spatial classification based on a combination of pixel-based and object-based scenarios

Earth Sci. Inform.

Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images

IEEE Trans. Pattern Anal. Mach. Intell.

Caractérisation de textures à l'aide d'un codage directionnel local

Afrique Science: Revue Internationale des Sciences et Technologie

Classification of urban areas from GeoEye-1 imagery through texture features based on Histograms of Equivalent Patterns

Eur. J. Remote Sens.

Melanoma-Pattern Extraction using Histogram-thresholding Approach

Off. J. Egypt. Soc. Med. Educ. Sci. Med. J. Sci. Med J.(ESCME)

Image segmentation by histogram thresholding using fuzzy sets

IEEE Trans. Image Process.

Active contours without edges

IEEE Trans. Image Process.

Segmentation d'images par contours actifs: application à la détection du ventricule gauche dans les images de scintigraphie cardiaque

SETIT

Active contour model based on local and global Gaussian fitting energy for medical image segmentation

Optik - Int. J. Light Electron Opt.