Do those who know more also know more about how much they know?

https://doi.org/10.1016/0030-5073(77)90001-0Get rights and content

Abstract

The validity of a set of subjective probability judgments can be assessed by examining two components of performance, calibration and resolution. The perfectly calibrated judge assigns probabilities so that, for all propositions assigned the same probability, the proportion true is equal to the probability assigned. For example, half of the propositions given a .50 chance of being true should in fact be true. Resolution reflects the degree to which assessors can successfully discriminate among different degrees of certainty, independent of the numerical labels assigned. A series of experiments revealed that: (1) Although people are moderately well calibrated, their probability judgments are prone to systematic biases. The most common bias is overconfidence. (2) People are calibrated differently when dealing with items of varying degrees of difficulty. (3) Calibration is unaffected by differences in intelligence, expertise, subjects' reliance on extreme probability responses, and at least some aspects of the context in which items are presented. (4) Resolution did not change as a function of difficulty, except for tasks about which subjects knew nothing. The implications of these results for decision makers are discussed.

References (30)

  • A. Tversky et al.

    Availability: a heuristic for judging frequency and probability

    Cognitive Psychology

    (1973)
  • J.K. Adams et al.

    Realism of confidence judgments

    Psychological Review

    (1961)
  • Atomic Energy Commission

    Reactor safety study: An assessment of accident risks in U.S. commercial power plants (WASH-1400 Draft)

    (1974)
  • G.W. Brier

    Verification of forecasts expressed in terms of probability

    Monthly Weather Review

    (1950)
  • F.R. Clarke

    Confidence ratings, second-choice responses, and confusion matrices in intelligibility tests

    Journal of the Acoustical Society of America

    (1960)
  • J. Cohen

    Chance, skill and luck: The psychology of guessing and gambling

    (1960)
  • W. Edwards et al.

    Decision making

    (1967)
  • N.T. Feather

    Subjective probability and decision under uncertainty

    Psychological Review

    (1959)
  • B. Fischhoff

    Cost-benefit analysis and the art of motorcycle maintenance

    Policy Sciences

    (1977)
  • Fischhoff, B., Slovic, P., & Lichtenstein, S. Knowing with certainty. Journal of Experimental Psychology: Human...
  • M. Fishbein

    A behavior theory approach to the relations between beliefs about an object and the attitude toward the object

  • E.E. Jones et al.

    From acts to dispositions: the attribution process in person perception

  • R. Kellogg

    Analyzing children's art

    (1970)
  • D.H. Krantz et al.
  • Lichtenstein, S., Fischhoff, B., & Phillips, L. D. Calibration of probabilities: the state of the art. In H. Jungermann...
  • Cited by (0)

    This research was supported by the Advanced Research Projects Agency of the Department of Defense and was monitored by the Office of Naval Research under Contracts NO0014-76-C-0074 (ARPA Order No. 2449) to Oregon Research Institute and Contract NO0014-76-C-0074 (ARPA Order No. 3052) under subcontract 76-030-0714 from Decisions and Designs, Inc., to Decision Research.

    View full text