Dynamic pattern denoising method using multi-basin system with kernels
Research highlights
► We propose a novel pattern denoising method using multi-basin dynamical systems. ► The constructed system characterizes the topological behavior of a noisy pattern. ► The denoised pattern is topologically most similar to the original noisy pattern. ► The method effectively reduces the noise while preserving the original information.
Introduction
In the past decade, learning techniques with kernels have been widely studied to solve various problems in machine learning and have expedited the development of many successful intelligent systems in various applications. Among them, support (or pseudo-density) learning methods with kernels have been successfully applied to pattern recognition tasks such as outlier or novelty detection [1], [2], data retrieval [3], [4], clustering [5], [6], [7], [8], [9], [14], [13] and classification [10], [16]. The advantages of these support learning methods with kernels over conventional support estimation methods are their flexibility to capture a support of arbitrary shape by utilizing the kernel mapping into the feature space and insensitivity to the outliers by following the structural risk minimization principle [6], [17], [14].
Generally, the role of a support function learned by existing methods that describes the domain of a data distributions is limited to separating novelties or outliers from normal patterns. However, when we have a prior knowledge that the outlier patterns are actually noised or corrupted version of normal patterns, we can reconstruct the denoised version of the noisy patterns by utilizing the domain information of normal patterns described by the support function and by associating outlier patterns with their corresponding normal patterns that are determined by the geometrical and topological affinities of the constructed support.
To accomplish this task, in this paper, we propose a novel pattern denoising method consisting of three steps: the first step to build a support function that describes a support of normal patterns, the second step to construct a so-called multi-basin system associated with the support function, and the final third step to denoise by moving given noisy pattern along the trajectory of multi-basin system. The denoised pattern obtained by this process is topologically most similar normal pattern with the original noisy pattern. We will show the effectiveness of the proposed method by demonstrating denoising performance on real image dataset such as USPS (United States Portal Service, [18]) handwritten digits, COIL20 object image library dataset [19] and C-Cube cursive character database [27] as well as a synthetic toy dataset to compare the results with various related methods.
The remainder of the paper is organized as follows: In Section 2, we provide the proposed dynamical denoising framework using a multi-basin system with theoretical analysis and give its detailed algorithms in Section 3. In Section 4, we demonstrate the effectiveness of the proposed method by comparing the experimental results with other denoising techniques in a number of toy and real-world problems. We conclude the paper in Section 5 with discussions.
Section snippets
The proposed method
Given a set of normal patterns , the proposed method, which is detailed below, consists of three phases; the first phase for constructing a support function, the second phase for building the multi-basin system associated with the support function, and the final third step for applying the system to the noisy pattern to obtain its denoised version.
Algorithm and implementation
The simplified algorithm for the proposed method is given as follows. Algorithm 1 Attracting Manifold Methods for Dynamical Pattern Denoising //A1. Training Support Function// Given a set of normal patterns and critical level value r Train a support function //A2. Constructing Multi-Basin System// Construct the multi-basin system associated with by defining Fq in (7). //A3. Numerical Integration for Denoising// For a given noisy pattern Set and t=0 while do Numerically
Experiments
In this section, we demonstrate the experimental results by applying the proposed method to simple toy example and real image denoising problems. We compare the denoising performance of the proposed method with other denoising methods both visually and quantitatively.
Conclusions
In this paper, we proposed a new denoising framework using topological behavior of a noisy pattern in the multi-basin system constructed by support functions. To do that, we first constructed a support function trained from a given normal dataset and built a multi-basin system associated with the learned support function. Then we moved a noisy pattern along the trajectory of the multi-basin system until it arrives to the boundary of the attracting manifold. Through the simulations, we
Acknowledgments
This work was supported partially by the Korea Research Foundation (KRF) Grant funded by the Korean Government (No. 2008-314-D00483) and partially by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2010-0025530).
Kyu-Hwan Jung received B.S. in industrial engineering from Pohang University of Science and Technology (POSTECH) in 2005 and is expected to receive a Ph.D. candidate in the Department of Industrial and Management Engineering at POSTECH in 2010. He is interested in pattern recognition, support vector machine, and their applications to data mining, pattern denoising, and image segmentation. E-mail: [email protected]
References (28)
- et al.
Support vector domain description
Pattern Recognition Letters
(1999) - et al.
Fast support-based clustering method for large-scale problems
Pattern Recognition
(2010) - et al.
Domain described support vector classifier for multi-class classification problems
Pattern Recognition
(2007) A SVM-based cursive character recognizer
Pattern Recognition
(2007)- et al.
Estimating the support of a high-dimensional distribution
Neural Computation
(2001) - C. Lai, D.M.J. Tax, R.P.W. Duin, E. Pekalska, P. Paclik, On combining one-class classifiers for image database...
- et al.
Non-relevance feedback document retrieval based on one-class SVM and SVDD
- et al.
Support vector clustering through proximity graph modelling
- et al.
Support vector clustering
Journal of Machine Learning Research
(2001) - et al.
An improved cluster labeling method for support vector clustering
IEEE Transactions on Pattern Analysis and Machine Intelligence
(2005)
Dynamic characterization of cluster structures for robust and inductive support vector clustering
IEEE Transactions on Pattern Analysis and Machine Intelligence
Equilibrium-based support vector machine for semi-supervised classification
IEEE Transactions on Neural Networks
Constructing sparse kernel machines using attractors
IEEE Transactions on Neural Networks
Dynamic dissimilarity measure for support-based clustering
IEEE Transactions on Knowledge and Data Engineering
Cited by (0)
Kyu-Hwan Jung received B.S. in industrial engineering from Pohang University of Science and Technology (POSTECH) in 2005 and is expected to receive a Ph.D. candidate in the Department of Industrial and Management Engineering at POSTECH in 2010. He is interested in pattern recognition, support vector machine, and their applications to data mining, pattern denoising, and image segmentation. E-mail: [email protected]
Namhyoung Kim received B.S. in industrial engineering from Pohang University of Science and Technology (POSTECH) in 2008 and is currently pursuing a Ph.D. in the Department of Industrial and Management Engineering at POSTECH. She is currently interested in pattern recognition, pattern denoising, and data mining. E-mail: [email protected]
Jaewook Lee is an associate professor in the Department of Industrial and Management Engineering at Pohang University of Science and Technology (POSTECH), Pohang, Korea. He received the B.S. degree from Seoul National University, and the Ph.D. degree from Cornell University in 1993 and 1999, respectively. His research interests include pattern recognition, neural networks, global optimization, nonlinear systems, and their applications to data mining and financial engineering. E-mail: [email protected]