doi:10.1016/j.patrec.2006.07.002
Copyright © 2006 Elsevier B.V. All rights reserved.
Improving nearest neighbor rule with a simple adaptive distance measure
aDepartment of Physics, The Institute for Brain and Neural Systems, Brown University, P.O. Box 1843, Providence, RI 02912, USA
Received 8 March 2006.
Communicated by R.P.W. Duin.
Available online 24 August 2006.
References and further reading may be available for this article. To view references and further reading you must
purchase this article.
Abstract
The k-nearest neighbor rule is one of the simplest and most attractive pattern classification algorithms. However, it faces serious challenges when patterns of different classes overlap in some regions in the feature space. In the past, many researchers developed various adaptive or discriminant metrics to improve its performance. In this paper, we demonstrate that an extremely simple adaptive distance measure significantly improves the performance of the k-nearest neighbor rule.
Keywords: Pattern classification; Nearest neighbor rule; Adaptive distance measure; Adaptive metric; Generalization error
Fig. 1. Error rates at different values of k on the five datasets. The plots are for the Wisconsin Breast Cancer, Ionosphere, Pima, Liver, and Sonar datasets in the top-down, left-right order respectively. Solid lines: the k-NN rule with the adaptive distance measure. Dashed lines: the k-NN rule with the Euclidean distance.
Fig. 2. Error rates at different values of k on the five datasets. Solid lines: the k-NN rule with the adaptive distance measure. Dashed lines: the k-NN rule with the Manhattan distance measure.
Fig. 3. The largest spheres associated with training examples that are inside the classes or near the class boundaries.
Fig. 4. Nearest neighbors according to different distance measures.
Fig. 5. Error rates at different values of k on the five datasets. The plots are for Breast Cancer, Ionosphere, Pima, Liver, and Sonar datasets in the top-down, left-right order respectively. Solid lines: k-NN rule with the adaptive distance measure. Dashed lines: k-NN rule with both the adaptive distance measure and the weighting scheme.
Table 1.
Comparison of error rates

Table 2.
Comparison of results
