Abstract
Standard setting extends the interpretations of scores by adding a standards-based inference (from test scores to performance levels) to the interpretation/use argument (IUA) for the underlying score scale. For standards-based interpretations and uses to be valid, this additional inference needs to be justified. The supporting evidence can be procedural, internal, and criterion-based. Criterion-based evidence is especially important in high-stakes contexts, where the standards tend to be contentious. Standards are inherently judgmental, and therefore, to some extent, arbitrary. The arbitrariness can be reduced to some extent by employing empirical relationships (e.g., dosage-response curves) to estimate upper and lower bounds on the cut score. In evaluating standards, the question is not whether we got it right, but rather, whether the decisions based on the cut scores are reasonable, broadly acceptable, and have mostly positive consequences (which outweigh any negative consequences).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Beaton, A., Linn, R., & Bohrnstedt, G. (2012). Alternative approaches to setting standards for the National Assessment of Educational Progress (NAEP). Sam Mateo: American Institutes for Research.
Campion, M. (1983). Personnel selection for physically demanding jobs: Review and recommendations. Personnel Psychology, 36, 527–550.
Cronbach, L. J. (1980). Validity on parole: How can we go straight? New directions for testing and measurement: Measuring achievement over a decade, 5, 99–108.
Cronbach, L. (1988). Five perspectives on validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Hillsdale: Erlbaum.
Fields, R. (2014). Towards the National Assessment of Educational Progress (NAEP) as an indicator of academic preparedness for college and job training. Washington, DC: National Assessment Governing Board.
Glass, G. V. (1978). Standards and criteria. Journal of Educational Measurement, 15, 237–261.
Greenberg, D. (1998, June 11). Of Human Poundage. The Lancet, 352, n 9122, p. 158.
Hambleton, R., & Pitoniak, M. (2006). Setting performance standards. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 433–470). Westport: Praeger Publishers.
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73.
Kane, M., Crooks, T., & Cohen, A. (1997, March). Justifying the passing scores for licensure and certification tests. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
Kane, M., Mroch, A., Ripkey, D., & Case, S. (2006). Impact of the increase in the passing score on the New York bar examination. Madison: National Conference of Bar Examiners http://www.nybarexam.org/NCBEREP.htm.
Loomis, S. C., & Bourque, M. L. (2001). National assessment of educational progress achievement levels, 1992–1998 for mathematics. Washington, DC: National Assessment Governing Board.
McLarty, K., Way, W., Porter, A., Beimers, J., & Miles, J. (2013). Evidence-based standard setting: establishing a validity framework for cut scores. Educational Researcher, 42(2), 78–88.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: American Council on Education and Macmillan.
Porter, T. (1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.
Shapiro, L. (1998, June 15). Fat, fatter: But who’s counting? Newsweek, 131(24), 55.
Shepard, L. (1980). Standard setting, issues and methods. Applied Psychological Measurement, 4, 447–467.
Toulmin, S. (2001). Return to reason. Cambridge, MA: Harvard University Press.
Tufts University. (2015). How much exercise is enough? Two new studies seek the ‘sweet spot’ for activity and intensity. Nutrition & Health Letter, 33(5), 7.
Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cut scores: A manual for setting standards of performance on educational and occupational tests. Princeton: Educational Testing Service.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Kane, M.T. (2017). Using Empirical Results to Validate Performance Standards. In: Blömeke, S., Gustafsson, JE. (eds) Standard Setting in Education. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-50856-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-50856-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50855-9
Online ISBN: 978-3-319-50856-6
eBook Packages: EducationEducation (R0)