doi:10.1016/j.patcog.2005.01.009
Copyright © 2005 Pattern Recognition Society Published by Elsevier B.V.
Exploring margin setting for good generalization in multiple class discrimination
References and further reading may be available for this article. To view references and further reading you must
purchase this article.
H. John Caulfielda and Kaveh Heidaryb, 
, 
aAlabama A&M University Research Institute, PO Box 313, Normal, AL 35762, USA
bDepartment of Electrical Engineering, Alabama A&M University, PO Box 702, Normal, AL 35762, USA
Received 8 October 2003;
revised 18 January 2005;
accepted 18 January 2005.
Available online 23 March 2005.
Abstract
In earlier publications, we showed that it is possible to achieve both low VC dimension and high accuracy, if we divide the given training set into a sequence of subsets each of which does admit such a solution. Here we explore in substantially more detail how the various steps in what was called “Margin Setting” impact false classification and indecision rates. A complex relationship exists between margin size, the number of steps in the process, and those two classification failures. After mapping those relationships, we offer a qualitative explanation of them.
Keywords: Pattern recognition; Classification; Margin; Generalization; Margin setting
Fig. 1. Classification circles corresponding to Table 1 (margin=10%).
Fig. 2. Percentage of correct classification and misclassification as functions of support margin using four classification rounds.
Fig. 3. Classification circles corresponding to margin=20%.
Fig. 4. Percentage of correct classification and misclassification as functions of support margin using six classification rounds.
Fig. 5. Percentage of correct classification and misclassification as functions of support margin using eight classification rounds.
Fig. 6. Percentage of correct classification and misclassification as functions of support margin using eight classification rounds.
Fig. 7. Percentage of correct classification and misclassification as functions of support margin using ten classification rounds.
Fig. 8. Percentage of correct classification and misclassification as functions of support margin using twelve classification rounds.
Fig. 9. Percentage of correct classification and misclassification as functions of support margin using twelve classification rounds. In each round, six generations of training vector mutations are used to determine the prototype.
Fig. 10. Percentage of correct classification and misclassification as functions of support margin using twelve classification rounds. In each round, six generations of training vector mutations are used to determine the prototype. Standard deviations for training and test samples is 0.15 (σx=σy=0.15).
Fig. 11. Percentage of correct classification and misclassification as functions of support margin using four classification rounds. In each round, six generations of training vector mutations are used to determine the prototype. Standard deviations for training and test samples is 0.15 (σx=σy=0.15).
Fig. 12. Percentage of correct classification and misclassification as functions of support margin using twelve classification rounds. In each round, six generations of training vector mutations are used to determine the prototype. Standard deviations for training and test samples is 0.25 (σx=σy=0.25).
Fig. 13. Classifiers for the three-class problem with x,y standard deviations of 0.25.
Fig. 14. Average classification error rate as a function of margin-value for the tow-class iris problem.
Table 1.
Classifiers with 10% margin evolved for four rounds or stages. Each is a circle centered at x,y with radius r
Table 2.
Results of tests with 10,000 A's.
Table 3.
Results of tests with 10,000 B's
Table 4.
Results of tests with 10,000 C's
Table 5.
Correct classification, misclassification, and non-classification rates for classifiers trained on and tested with the Fisher iris data
Table 6.
Characteristics of a 10 trainer 10% margin classifier
Corresponding author. Tel.: +1 256 372 5587; fax: +1 256 372 5855.
Abstract
Purchase PDF (1133 K)
E-mail Article
Add to my Quick Links
Cited By in Scopus (3)
Purchase PDF (221 K)
