Abstract
Although the weights in a discriminant function (both linear and quadratic) are independent of group prior probabilities, the performance of the classifier (on the training and validation data) is sensitively dependent on these often unknown probabilities. After reviewing some defects of a popular measure of performance in the situation where the group sizes are naturally disproportionate, three alternative measures of performance (or association) are considered and it is shown that the behavior of the measures as a function of group prior probability is different between measures. Consequently, the optimum choice of the group prior probability depends on the specific measure of performance. Among the measures considered, only two measures - the index of mean square contingency and the Heidke Skill Statistic - are found to be well defined in the disparate-group size situation, and are, therefore, recommended. An empirical data set, dealing with delinquency among high school students is employed to illustrate all of the findings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderer, P., Saletu, B., Klöppel, B., Semlitsch, H. V. & Werner, H. (1994). Discrimination between demented patients and normals based on topographic EEG slow wave activity: comparison between z statistics, discriminant analysis and artificial neural network classifiers, Electroencephalography and Clinical Neurophysiology 91: 108–117.
Anson, O. & Sagy, S. (1995). Marital violence: Comparing women in violent and nonviolent unions, Human Relations 48: 285–305.
Azari, N. P., Pietrini, P., Horwitz, B. & Pettigrew, K. D. (1993). Individual differences in cerebral metabolic patterns during pharmacotherapy in obsessive-compulsive disorder: A multiple regression/discriminant analysis of positron emission tomographic data, Biological Psychiatry 34: 798–809.
Bellantoni, L., Conway, J. S., Jacobsen, J. E., Pan, Y. B. & Wu, S. L. (1991). Using neural networks with jet shapes to identify b jets in e + e − interactions, Nuclear Instruments and Methods A-310: 610–615.
Bernard, L. C., McGrath, M. J. & Houston, W. (1993). Discriminating between simulated malingering and closed head injury on the Wechsler Memory Scale — revised, Archives of Clinical Neuropsychology 8: 539–551.
Boone, S. L. (1991). Aggression in African-American boys: A discriminant analysis. Genetic, Social, and General Psychology Monographs, 117: 203–228.
Bowser-Chao, D. & Dzialo, D. L. (1993). Comparison of the use of binary decision trees and neural networks in top-quark detection, Physical Review D 47: 1900–1905.
Camilli, G. (1990). The test of homogeneity for 2 × 2 contingency tables: A review of and some personal opinions on the controversy, Psychological Bulletin 108: 135–145.
Cherry, A. (1993). Combining cluster and discriminant analysis to develop a social bond typology of runaway youth, Research on Social Work Practice 3: 175–190.
Christensen, L. & Duncan, K. (1995). Distinguishing depressed from nondepressed individuals using energy and psychosocial variables, Journal of Consulting and Clinical Psychology 63: 495–498.
Dammers, E. (1993). Measurement in the ex post evaluation of forecasts, Quality and Quantity 27: 31–45.
Dannehl, C. R. & Groth, A. J. (1992). Communist and non-communist Europe: Functional differentiation, 1970–1985, Social Indicators Research 27: 59–87.
Doolittle, M. H. (1888). Association ratios, Bulletin of the Philosophical Society of Washington 10: 83–87, 94–96.
Famularo, R., Fenton, T., Kinscherff, R., Barnum, R., Bolduc, S. & Bunschaft, D. (1992). Differences in neuropsychological and academic achievement between adolescent delinquents and status offenders, American Journal of Psychiatry 149: 1252–1257.
Fienberg, S. E. (1980). The Analysis of Cross-Classified Categorical Data (2nd edn). Cambridge, MA: MIT Press.
Glick, N. (1978). Additive estimators for probabilities of correct classification, Pattern Recognition 10: 211–222.
Goodman, L. A. & Kruskal, W. H. (1954). Measures of association for cross classifications, American Statistical Association Journal 49: 723–764.
Goodman, L. A. & Kruskal, W. H. (1959). Measures of association for cross classifications. II: Further discussion and references, American Statistical Association Journal 54: 123–163.
Haber, M. (1990). Comments on “the test of homogeneity for 2 × 2 contingency tables: A review of and some personal opinions on the controversy” by G. Camilli, Psychological Bulletin 108: 146–149.
Hammond, S. M. & Lienert, G. A. (1995). Modified Phi correlation coefficients for the multivariate analysis of ordinally scaled variables, Educational and Psychological Measurement 55: 225–236. Hays, W. L. (1973). Statistics for the Social Sciences (2nd edn). New York, NY: Holt, Rinehart and Winston.
Huberty, C. A. (1994). Applied Discriminant Analysis. New York, NY: John Wiley & Sons, Inc.
Klecka, W. R. (1980). Discriminant Analysis. Newbury Park, CA: Sage Publications.
Lachenbruch, P. A. (1975). Discriminant Analysis. New York, NY: Hafner Press.
Marzban, C. (1997). Scalar Measures of Performance in Rare-event Situations, To appear in Weather and Forecasting.
McLachlan, G. J. (1992). Discriminant Analysis and Statistical Pattern Recognition. New York, NY: A Wiley-Interscience Publication.
Morwitz, V. G. & Schmittlein, D. (1992). Using segmentation to improve sales forecasts based on purchase intent: Which “intenders” actually buy?, Journal of Marketing Research 29: 391–405.
Murphy, A. H. & Daan, H. (1985). Forecast Evaluation, in A. H. Murphy & R. W. Katz (eds), Probability, Statistics and Decision Making in the Atmospheric Sciences, pp. 379–437. Boulder, CO: Westview Press.
National Opinion Research Center. (1980). High School and Beyond Information for Users: Base Year (1980) Data. Chicago: Author.
Ott, L., Larson, R. F. & Mendenhall, W. (1983). Statistics: A Tool for the Social Sciences (3rd edn). Boston, MA: Duxbury Press.
Paik, H. & Comstock, G. (1994). The effects of television violence on antisocial behavior: A meta-analysis, Communication Research 21: 516–546.
Paik, H. & Marzban, C. (1995). Predicting television extreme viewers and nonviewers: A neural network analysis, Human Communication Research 22: 284–306.
Parshall, C. G. & Kromrey, J. D. (1996). Tests of independence in contingency tables with small samples: A comparison of statistical power, Educational and Psychological Measurement 56: 26–44.
Peirce, C. S. (1884). The numerical measure of the success of predictions [Letter to the editor], Science 4: 453–454.
Reiss, A. J. & Roth, J. A. (eds). (1994). Understanding and Preventing Violence, Vol. 3: Social Influences. Washington, DC: National Academy Press.
SAS Institute Inc. (1989). SAS/STAT® User's Guide, Version 6, Fourth Edition, 1, Cary, NC: SAS Institute Inc.
Stimpfl-Abele, G. (1991). Recognition of decays of charged tracks with neural network techniques, Computer Physics Communications 67: 183–192.
Wilson, R. L. & Hardgrave, B. C. (1995). Predicting graduate student success in an MBA program regression versus classification, Educ. Psych. Measurement 55: 186–195.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Paik, H. The Effect of Prior Probability on Skill in Two-Group Discriminant Analysis. Quality & Quantity 32, 201–211 (1998). https://doi.org/10.1023/A:1004359127048
Issue Date:
DOI: https://doi.org/10.1023/A:1004359127048