Abstract
Carleton University, Ottawa, Ontariot Canada Confidence rating based calibration and resolution indices were obtained in two experiments requiring perceptual comparisons and in a third with visual gap detection. Four important results were obtained. First, as in the general knowledge domain, subjects were underconfident when judgments were easy and overconfident when they were difficult. Second, paralleling the clear dependence of calibration on decisional difficulty, resolution decreased with increases in decision difficulty arising either from decreases in discriminability or from increasing demands for speed at the expense of accuracy. Third, providing trial-by-trial response feedback on difficult tasks improved resolution but had no effect on calibration. Fourth, subjects can accurately reportsubjective errors (i.e., trials in which they have indicated that they made an error) with their confidence ratings. It is also shown that the properties of decision time, conditionalized on confidence category, impose a rigorous set of constraints on theories of confidence calibration.
Article PDF
Similar content being viewed by others
References
Adams, J K (1957) A confidence scale defined in terms of expected percentagesAmerican Journal of Psychology,70. 432–436
Adams, J. K., &Adams, P. A (1961). Realism of confidence judgmentsPsychological Review,68, 33–45.
Adams, P. A., &Adams, J. K. (1958) Training in confidence-judgments.American Journal of Psychology,71, 747–751.
Allan, L. G. (1979) The perception of timePerception & Psychophysics,26, 340–354.
Allwood, C. M., &Montgomery, H (1987). Response selection strategies and realism of confidence judgmentsOrganizational Behavior & Human Decision Processes.39, 365–383,
Arkes, H R, Christensen, C, Lai, C, &Blumer, C. (1987). Two methods of reducing overconfidence.Organizational Behavior & Human Decision Processes,39, 135–144
Audley, R. J (1960) A stochastic model for individual choice behavior.Psychological Review,67, 1–15.
Baranski, J. V. (1991).Theories of confidence calibration and experiments on the time to determine confidence. Unpublished doctoral dissertation, Carleton University, Ottawa, ON
Baranski, J. V., & Petrusic, W. M. (1991)Confidence calibration and the scaling of doubt. Paper presented at the 52nd annual meeting of the Canadian Psychological Association, Calgary, AB.
Baranski, J. V., &Petrusic, W. M. (1992). The discriminatory of remembered magnitudes.Memory & Cognition,20, 254–270.
Baron, J. (1988).Thinbng and deciding. Cambridge: Cambridge University Press
Bjork, E. L., &Murray, J. T. (1977). On the nature of input channels in visual processing.Psychological Review,84, 472–484.
Bjorkman, M., Juslin, P., &Wjnman, A. (1993) Realism of confidence in sensory discrimination The underconfidence phenomenon.Perception & Psychophysics,54, 75–81.
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability.Monthly Weather Review,78, 1–3.
Carlson, B. W. (1993) The accuracy of future forecasts and past judgmentsOrganizational Behavior & Human Decision Processes,54, 245–276.
Christensen-Szalanski, J., &Bushyhead, J. (1981). Physicians use of probabilistic information in a real clinical setting.Journal of Experimental Psychology. Human Perception & Performance,7,928–935
Coombs, C. H. (1964)A theory of data. New York; Wiley,
Dawes, R. M. (1980). Confidence in intellectual vs. confidence in perceptual judgments. In E. D. Lantermann & H Feger (Eds.).Similarity and choice: Papers in honor of Clyde Coombs (pp. 327–345). Bern Hans Huber
De Finetti, B, (1937) La prevision: Ses lois logiques, ses sources subjectives.Annales de l’Institut Henri Potncaré,7, 1–68 English translation in H E. Kyburg, Jr., & H. E. Smokier (Eds ) (1964),Studies in subjective probability (pp. 93-158) New York: Wiley
Egeth, H. E., &Santel, J. L. (1981) Conceptual and perceptual components of interletter inhibition.Journal of Experimental Psychology: Human Perception & Performance,7, 506–517
Ferrell, W. R., &McGoey, P J (1980) A model of calibration for subjective probabilities.Organizational Behavior & Human Performance,26, 32–53.
Festinger, L. (1943). Studies in decision: I. Decision-time, relative frequency of judgment and subjective confidence as related to physical stimulus difference.Journal of Experimental Psychology,32, 291–306
Fischhoff, B (1982). Debiasing In D. Kahneman, P. Slovic, & A Tversky (Eds.),Judgments under uncertainty. Heuristics and biases (pp. 422–444). Cambridge: Cambridge University Press
Fischhoff, B., &Macgregor, D. (1982) Subjective confidence in forecasts.Journal of Forecasting,1, 155–172.
Fischhoff, B., Slovic, P., &Lichtfnstein, S. (1977). Knowing with certainty: The appropriateness of extreme confidenceJournal of Experimental Psychology: Human Perception & Performance,3. 552–564
Fullerton, G. S., &Cattell, J. M. (1892).On the perception of small differences Philadelphia. University of Pennsylvania Press
Garrett, H. E. (1922). A study of the relation of accuracy to speed.Archives of Psychology.56, 3–104.
Geller, E. S., &Pitz, G. F. (1968) Confidence and decision speed in the revision of opinion.Organizational Behavior & Human Performance,3, 190–201.
Gigerenzer, G., Hoffrage, U., &Kleinbölting, H. (199T) Probabilistic mental models A Brunswikian theory of confidencePsychological Review.98, 506–528
Glenbfrg, A. M., &Epstein, W. (1985) Calibration of comprehensionJournal of Experimental Psychology: Learning, Memon, & Cognition,11, 702–718
Glenberg.A. M., &Epstein, W (1987). Inexpert calibration of comprehensionMemory & Cognition,15, 84–93,
Glenberg, A. M., Samocki, T., Epstein, W., &Morris, C. (1987) Enhancing calibration of comprehensionJournal of Experimental Psychology: Generai,116, 119–136
Green, D. M., &Swets, J. A. (1966).Signal detection theory and psychophysics. New York Wiley
Griffin, D., &Tversky, A. (1992) The weighing of evidence and the determinants of confidenceCognitive Psychology,24, 411 -435
Heath, R. A. (1984). Random-walk and accumulator modeis of psychophysical discrimination A critical evaluation.Perception,13, 57–65.
Heelstrom, A. (1985) The time-order error and its relatives: Mirrors of cognitive processes in comparing.Psychological Bulletin.97, 35–61.
Henmon.V. A. C. (1911). The relation of the time of a judgment to its accuracyPsychological Review,18, 186–201.
Irwin, F. W., Smith, W. A. S., &Mayfteld, J. F. (1956) Tests of two theories of decision in an “expanded judgment” situationJournal of Experimental Psychology,51. 261–268.
Jamieson, D. G., &Petrusic, W. M. (1975) Presentation order effects in duration discrimination.Perception & Psychophysics,17, 197–202
Johvson, D. M. (1939). Confidence and speed in the two-category judgmentArchives of Psychology,34, 1–53.
Keren, G. (1987). Facing uncertainty in the game of bridge; A calibration study.Organizational Behavior & fiumun Decision Processes,39, 98–114.
Keren, G. (1988). On the ability of monitoring non-veridical perceptions and uncertain knowledge: Some calibration studies.Acta Psychohgica,67, 95–119.
Keren, G. (1991) Calibration and probability judgments. Conceptual and methodological issuesActa Psychologica,77, 217–273.
Keren, G., &Wacenaar, W. A. (1987). Temporal aspects of probabilistic predictionsBulletin of the Psychonomic Society,25, 61–64
Koriat, A., Lichtenstein, S., &Fischhoff, B. (1980). Reasons for confidence.Journal of Experimental Psychology. Human Learning & Memory.6, 107–118
Liberman, V., &Tversky, A. (1993) On the evaluation of probability judgments: Calibration, resolution and monotonicity.Psychological Bulletin,114, 162–173
Lichtenstein.S., &Fischhoff, B. (1977). Do those who know more also know more about how much they know? The calibration of probability judgments.Organizational Behavior & Human Performance,20, 159–183
Lichtenstein, S. &Fischhoff, B. (1980) Training for calibration.Organizational Behavior & Human Performance,26, 149–171.
Lichtevstein, S., Fischhoff, B., &Phillips, L. D. (1982). Calibration of probabilities. The state of the art to 1930 In D. Kahnemann, P. Slovic, & A. Tversky (Eds),Judgment under uncertainty: Heuristics and biases (pp. 306–334) Cambridge: Cambridge University Press
Link, S W. (1975). The relative judgment theory of two choice response time.Journal of Mathematical Psychology,12, 114–135.
Link, S. W. (1992).The wave theory ofdifference and similarity Hillsdale, NJ. Erlbaum.
Link, S. W., &Heath, R. A. (1975) A sequential theory of psychological discriminationPsychometrika.40, 77–105,
Luce, R. D. (1986)Response times. New York: Oxford University Press.
Lund, F. H. (1926). The criteria of confidenceAmerican Journal & Psychology.37, 372–381.
Maki, R. H. &Berry, S. L. (1984) Metacomprehension of text material.Journal of Experimental Psychology. Learning. Memory, & Cognition,10, 663–679
Makl, R. H. &Swfti, S. (1987) Metamemory for narrative text.Memory & Cognition,15, 72–83
May, R. S. (1986) Inferences, subjective probability and frequency of correct answers A cognitive approach to the overconfidence phenomenon In B. Brehmer, H. Jungermann, & G. Sevon (Eds.).New directions in research on decision making (pp. 175–189). Amsterdam. Elsevier.
Morris, C. C. (1990) Retrieval processes underlying confidence in comprehension judgments.Journal of Experimental Psychology: Learning, Memory, Cognition,16, 223–232
Munsterberg, H. (1894). Studies from the Harvard psychological laboratory A psychometric investigation of the psychophysic law.Psychological Review,1, 45–51
Murdock.B. B., Jr., &Dufty, P. O. (1972) Strength theory and recognition memory.Journal of Experimental Psychology,94. 284–290,
Murphy, A. H. (1972). Scalar and vector partitions of the probability score: Part 1. Two-state situation.Journal of Applied Meteorology,11, 273–282
Murphy, A. H. (1973) A new vector partition of the probability score.Journal of Applied Meteorology12, 595–600
Murphy, A. H., &Winkler, R. L. (1977). Can weather forecasters formulate reliable probability forecasts of precipitation and temperature?National Weather Digest,2, 2–9.
Myers, J. L., &Well, A. D. (1991).Research design & statistical analysis New York: Harper Collins.
Nickerson, R. S., &McGoldrick, C. C. (1963). Confidence, correctness, and difficulty with non-psychophysical comparative judgments.Perceptual & Motor Skills,17, 159–167
Nickfrson, R. S., &McGoldrick, C. C. (1965) Confidence ratings and level of performance on a judgmental taskPerceptual & Motor Skills,20, 311–316.
Norman, D. A., &Wickflgren, W. A. (1969), Strength theory of decision rules and latency in short-term memory.Journal of Mathematical Psychology,6, 192–208.
O’connor, M., &Lawrence, M. (1989), An examination of the accuracy of judgmental confidence intervals in time series forecastingJournal of Forecasting,8, 141–155
Oscamp.S. (1965), Overconfidence in case-study judgments.Journal of Consulting Psychology,29, 261–265.
Peirce, C. S., &Jastrow, J. (1884) On small differences of sensation.Memoirs of the National Academy of Sciences,3, 75–83
Petrusic, W. M. (1992). Semantic congruity effects and theories of the comparison process.Journal of Experimental Psychology, Human Perception & Performance,18, 962–986
Petrusic, W. M., &Baranski, J. V. (1989a). Context, context shifts, and semantic congruity effects in comparative judgments. In D. Vickers & P. Smith (Eds.),Human information processing: Measures, mechanisms, and models (pp. 231–251). Amsterdam: Elsevier.
Petrusic, W. M., &Baranski, J. V. (1989b). Semantic congruity effects in perceptual comparisons.Perception Psychophvsics,45, 439–452
Petrusic, W M., &Jamieson, D. G. (1978) The relation between probability of preferential choice and the time to choose changes with practiceJournal of Experimental Psychology: Human Perception & Performance,4. 471–482
Petrusic, W. M., &Jamieson, D. G. (1979) Resolution time and the coding of arithmetic relations on supraliminally different visual extents.Journal of Mathematical Psychology,19, 89–107
Phillips, L. D. (1973)Bayesian statistics for social scientists London. Nelson.
Pierrel.R., &Murray, C. S. (1963) Some relationships between comparative judgment, confidence, and decision-time in weight-lifting,American Journal of Psychology,76. 28–38
Pike, A. R. (1968) Latency and relative frequency of response in psychophysical discriminationBritish Journal of Mathematical and Statistical Psychology,21, 161–182
Pike, A. R (1971) The latencies of correct and incorrect responses in discrimination and detection tasks. Their interpretation in terms of a model based on simple countingPerception & Psychophysics,9, 455–460
Pike, A. R. (1973) Response latency mechanisms for signal detection.Psychological Review,80, 53–68
Rabbitt, P. M. A., &Vyas, S. M. (1970) An elementary preliminary taxonomy for some errors in laboratory choke RT tasksActa Psychologica,33. 56–76
Ronis, D. L., &Yates, J. F. (1987). Components of probability judgment accuracy: Individual consistency and effects of subject matter and assessment method.Organizational Behavior & Human Decision Processes,40, 193–218.
Sanders, F. (1963). On subjective probability forecasting.Journal of Applied Meteorology,2, 191–201.
Savage, L.J. (1954).The foundations of statistics. New York: Wiley.
Seward, G. H. (1928). Recognition time as a measure of confidence.Archives of Psychology,99, 2–54.
Sharp, G. L., Cutler, B. L., &Penrod, S. D. (1988). Performance feedback improves the resolution of confidence judgments.Organizational Behavior & Human Decision Processes,42, 271–283.
Smith, P. L., &Vickers, D. (1988). The accumulator model of two choice discrimination.Journal of Mathematical Psychology,32, 135–168.
Solomon, I., Ariyo, A., &Tomassini, L. A. (1985). Contextual effects on the calibration of probabilistic judgments.Journal of Applied Psychology,70, 528–532.
Tanner, W. P., &Swets, J. A. (1954). A decision making theory of visual detection.Psychological Review,61, 401–409.
Thomas, E. A. C. (1971). Sufficient conditions for monotone hazard rate: An application to latency-probability curves.Journal of Mathematical Psychology,8, 303–332.
Tomassini, L. A., Solomon, I., Romney, M. B., &Krogstad, J. L. (1982). Calibration of auditors’ probabilistic judgments: Some empirical evidence.Organizational Behavior & Human Performance,30, 391–406.
Trow, W. C. (1923). The psychology of confidence.Archives of Psychology,67, 1–47.
Vickers, D. (1970). Evidence for an accumulator model of psychophysical discrimination.Ergonomics,13, 37–58.
Vickers, D. (1979).Decision processes in visual perception. New York: Academic Press.
Vickers, D., &Packer, J. (1982). Effects of alternating set for speed versus accuracy on response time, accuracy, and confidence in a unidimensional discrimination task.Acta Psychologica,50, 179–197.
Vickers, D., Smith, P., Burt, J., &Brown, M. (1985). Experimental paradigms emphasizing state or process limitations: II. Effects on confidence.Acta Psychologica,59, 163–193.
Volkmann, J. (1934). The relation of the time of judgment to the certainty of judgment.Psychological Bulletin,31, 672–673.
Vreugdenhil, H., &Koele, P. (1988). Underconfidence in predicting future events.Bulletin of the Psychonomic Society,26, 236–237.
Wagenaar, W. A. (1988). Calibration and the effects of knowledge and reconstruction in retrieval from memory.Cognition,28, 277–296.
Wagenaar, W. A., &Keren, G. (1985). Calibration of probability assessments by professional blackjack dealers, statistical experts, and lay people.Organizational Behavior & Human Decision Processes,36, 406–416.
Weaver, C. A. (1990). Constraining factors in calibration of compre-hension.Journal of Experimental Psychology: Learning, Memory, & Cognition,16, 214–222.
Winman, A., &Juslin, P. (1993). Calibration of sensory and cognitive judgments: Two different accounts.Scandinavian Journal of Psychology,34, 135–148.
Wright, G. (1982). Changes in the realism and distribution of probability assessments as a function of question type.Acta Psychologica,52, 165–174.
Wright, G., &Ayton, P. (1986). Subjective confidence in forecasts: A response to Fischhoff and MacGregor.Journal of Forecasting,5, 117–123.
Wright, G., &Phillips, L. D. (1980). Cultural variation in probabilistic thinking: Alternative ways of dealing with uncertainty.International Journal of Psychology,15, 239–257.
Wright, G., &Wisudha, A. (1982). Distribution of probability assessment for almanac and future event questions.Scandinavian Journal of Psychology,23, 219–224.
Yaniv, I., Yates, J. F., &Smith, J. E. K. (1991). Measures of discrimination skill in probabilistic judgment.Psychological Bulletin,110,611–617.
Yates, J. F. (1982). External correspondence: Decompositions of the mean probability score.Organizational Behavior & Human Decision Processes,30, 132–156.
Yates, J. F. (1990).Judgment and decision making. Englewood Cliffs, NJ: Prentice-Hall.
Author information
Authors and Affiliations
Additional information
This research was funded by a grant from the Natural Sciences and Engineering Research Council of Canada to W.M P A portion of the data reported in Experiment I was presented at the annual meeting of the Canadian Psychological Association. Montreal. June 1988.
Rights and permissions
About this article
Cite this article
Baranski, J.V., Petrusic, W.M. The calibration and resolution of confidence in perceptual judgments. Perception & Psychophysics 55, 412–428 (1994). https://doi.org/10.3758/BF03205299
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03205299