Skip to main content

Advertisement

SpringerLink
Log in

All Bachelors are Unmarried Men (p <  0.05)

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

This paper adds to the list of criticisms against null hypothesis significance testing (NHST). I argue that when researchers do not analyze the conceptual relations among their variables, they may fail to distinguish between logical implications and empirical relations. It does not make sense to use significance testing on hypotheses involving conceptually related phenomena. The widespread lack of conceptual clarification also leads to very small effect sizes in psychology because it causes study participants to understand the stimulus material in different ways. Therefore, they answer in an inconsistent way. Researchers show an extremely low degree of ambition when they seek to show that psychological phenomena differ from chance, or when they try to disprove a hypothesis claiming that a psychological phenomenon does not exist. I see significance testing as a poor solution to the problem of tiny effect sizes in psychology. I recommend that psychological researchers be more explicit both about their main hypotheses and their auxiliary hypotheses. As examples, I analyse all quantitative articles in Issue 1, 2005 of the Journal of Health Psychology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bagozzi R.P., Ascione F.J., Mannebach M.A. (2005). Inter-role in hospital-based pharmacy and therapeutics committee decision making. Journal of Health Psychology 10: 45–64

    Article  Google Scholar 

  • Bem D.J. (1996) Ganzfeld phenomena. In: Stein G (eds). Encyclopedia of the Paranormal. Prometheus Books, Buffalo NY, pp. 291–296

  • Berkson J. (1938). Some difficulties of interpretation encountered in the application of the chi-square test. Journal of the American Statistical Association 33: 526–542

    Article  Google Scholar 

  • Cohen J. (1990). Things I have learned (so far). American Psychologist 45: 1304–1312

    Article  Google Scholar 

  • Cohen J. (1994). The earth is round (p < .05). American Psychologist 49: 997–1003

    Article  Google Scholar 

  • Falk R., Greenbaum C.W. (1995). Significance tests die hard: the persistence of a probabilistic misconception. Theory and Psychology 5: 75–98

    Article  Google Scholar 

  • Harlow L.L., Mulaik S.A., Steiger J.H. (1997). What if there were no Significance Tests?. Mahwah NJ, Lawrence Erlbaum Associates

    Google Scholar 

  • Heider F. (1958). The Psychology of Interpersonal Relations. New York, Wiley

    Google Scholar 

  • Helstrup T., Rognes W., Vollmer F. (1999). Psychologic and memory. Scandinavian Journal of Psychology 40(Suppl. 1): 1–138

    Google Scholar 

  • Hubbard R., Parsa R.A., Luthy M.R. (1997). The spread of statistical significance testing in psychology. The case of the Journal of Applied Psychology, 1917–1994. Theory and Psychology 7: 545–554

    Google Scholar 

  • Ingledew D.K., Wray J.L., Markland D., Hardy L. (2005). goal perceptions and affective well-being. Journal of Health Psychology 10: 101–122

    Article  Google Scholar 

  • Insel K.C., Meek P.M., Leventhal H. (2005). Differences in illness representation among pulmonary patients and their providers. Journal of Health Psychology 10: 147–162

    Article  Google Scholar 

  • Iwasaki Y., Mannell R., Smale B.J., Butcher J. (2005). of leisure participation in predicting stress coping and health among police and emergency response service workers. Journal of Health Psychology 10: 79–99

    Google Scholar 

  • Jones L. (1955). Statistics and research design. Annual Review of Psychology 6: 405–430

    Article  Google Scholar 

  • Kahneman, D. Tversky, A. (1990). Prospect theory: an analysis of under risk. In: P. K. Moser (ed.), Rationality in Action: Approaches vol. New York, NY: Cambridge University Press, pp. 140–170

  • Kenny D. (1994). Interpersonal Perception: A Social Relations Analysis. New York, Guilford Press

    Google Scholar 

  • Kish L. (1959). Some statistical problems in research design. American Sociological Review 24: 328–338

    Article  Google Scholar 

  • Krueger J. (2001). Null hypothesis significance testing: on the survival of a flawed method. American Psychologist 56: 16–26

    Article  Google Scholar 

  • Kukla A. (1989). Nonempirical issues in psychology. American 44: 785–794

    Google Scholar 

  • Marks D.F. (2000a). A pragmatic basis for judging models and theories in health psychology: the axiomatic method Open peer commentary. Journal of Health Psychology 10(1): 5–176

    Article  Google Scholar 

  • Marks D.F. (2000b). Editorial. Journal of Health Psychology 5: 131–132

    Article  Google Scholar 

  • Marks, D. F. (ed.) (2005). Journal of Health Psychology 10(1).

  • Mccabe M.P., Judicibus M.D. (2005). The effects of economic disadvantage on psychological well-being and quality of life among people with multiple sclerosis. Journal of Health Psychology 10: 163–173

    Article  Google Scholar 

  • Meehl P.E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology 46: 806–834

    Article  Google Scholar 

  • Meehl P.E. (1997). The problem is epistemology, not statistics: replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In: Harlow L., Mulaik S., Steiger J. (eds). What if There were no Significance Tests?. Erlbaum, Mahwah, NJ, pp. 393–425

    Google Scholar 

  • Oakes M. (1986). Statistical Inference: A Commentary for the Social and Behavioral Sciences. New York, Wiley

    Google Scholar 

  • Ogden J. (2003). Some problems with social cognition models: a pragmatic and conceptual analysis. Health Psychology 22: 431–434

    Article  Google Scholar 

  • Ossorio P.G. (1991). Naive baseball theory. Psychological Inquiry 2: 352–355

    Article  Google Scholar 

  • Parrot W.G., Harré R. (1991). Smedslundian suburbs in the city of the case of embarrassment. Psychological Inquiry 2: 358–360

    Article  Google Scholar 

  • Pervin L. (1991). The pseudoempirical in psychology and the case for Psychologic. Psychological Inquiry 2(4): 325–382

    Article  Google Scholar 

  • Putnam, H. (1975). The analytic and the synthetic. In: H. Putnam (ed.), Philosophical Papers: Mind, Language and Reality, vol. 2. Cambridge: Cambridge University Press, pp. 33–69.

  • Rivers S.E., Salovey P., Pizarro D.A., Pizarro J., Schneider T.R. (2005). Message framing and pap test utilization among women attending a community health clinic. Journal of Health Psychology 10: 65–77

    Article  Google Scholar 

  • Rothman K.J., Greenland S. (1998). Approaches to statistical analysis. In: Rothman K.J., Greenland S. (eds). Modern Epidemiology, 2nd edn. Lippincott Williams,sss Wilkins, London pp. 183–199

  • Rozeboom W.W. (1960). The fallacy of the null-hypothesis significance test. Psychological Bulletin 67: 416–428

    Article  Google Scholar 

  • Schmidt F.L. (1996). Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods 1: 115–129

    Article  Google Scholar 

  • Shotter J. (1991). Measuring blindly and speculating loosely: But is a the answer?. Psychological Inquiry 2: 363–365

    Article  Google Scholar 

  • Shweder R.A. (1991). On pseudoempiricism, pseudodeductionism, and common sense. Psychological Inquiry 2: 366–370

    Article  Google Scholar 

  • Smedslund G. (2000). A pragmatic basis for judging models and theories in health psychology: The axiomatic method (target paper). Journal of Health Psychology 5: 133–149

    Google Scholar 

  • Smedslund J. (1978). Bandura’s theory of self-efficacy: a set of common sense theorems. Scandinavian Journal of Psychology 19: 1–14

    Google Scholar 

  • Smedslund, J. (1984). What is necessarily true in psychology? In: J. R. Royce and L. P. Mos (eds.), Annals of Theoretical Psychology. Plenum Press, New York London, pp. 241–272

  • Smedslund J. (1987a). Ebbinghaus, the illusionist: how psychology came to look like an experimental science. Passauer Schriften zur Psychologiegeschichte 5: 225–239

    Google Scholar 

  • Smedslund J. (1987b). The epistemic status of inter-item correlations in Eysenck’s Personality Questionnaire: the a priori versus the empirical in psychological data. Scandinavian Journal of Psychology 28: 42–55

    Google Scholar 

  • Smedslund J. (1988). Psycho-Logic. Springer Verlag, Berlin Heidelberg New York

    Google Scholar 

  • Smedslund J. (1991). The pseudoempirical in psychology and the case for Psychologic. Psychological Inquiry 2: 325–338

    Article  Google Scholar 

  • Smedslund J. (1995). Auxiliary versus theoretical hypotheses and ordinary versus scientific language. Human Development 38: 174–178

    Article  Google Scholar 

  • Smedslund, J. (1997a). Is the ‘psychologic’ of trust universal? In: Niemeier S., Dirven R. (eds) The Language of Emotions. Conceptualization, Expression, and Theoretical Foundation. Amsterdam/Philadelphia: John Benjamins Publishing Company, pp. 3–13.

  • Smedslund J. (1997b). The Structure of Psychological Common Sense. Lawrence Erlbaum Associates Publishers, Mahwah, New Jersey, London

    Google Scholar 

  • Smedslund J. (1999). Psychologic and the study of memory. Scandinavian Journal of Psychology 40: 3–17

    Google Scholar 

  • Smedslund J. (2002). From hypothesis-testing psychology to procedure-testing psychologic. Review of General Psychology 6: 51–72

    Article  Google Scholar 

  • Smyth M.M. (2001). Fact making in psychology: the voice of the textbook. Theory and Psychology 11: 609–636

    Article  Google Scholar 

  • Sterne, J. A. & Davey Smith, G. (2001). Sifting the evidence – what’s wrong with significance tests? British Medical Journal 322(January): 226–231

    Google Scholar 

  • Storm, L. & Ertel, S. (2001). Does psi exist? Comments on Milton and Wiseman’s (1999): meta-analysis of ganzfeld research. Psychological Bulletin 127: 424–433.

    Google Scholar 

  • The Free Dictionary (2005). Retrieved December 21, 2005, from http:// www. thefreedictionary.com/helplessness

  • Wallach L., Wallach M.A. (1999). Why is experimentation in psychology often senseless?. Scandinavian Journal of Psychology 40: 103–106

    Google Scholar 

  • Wallach M.A., Wallach L. (1998a). Of surrogacy, circularity, causality and near-tautologies: a response. Theory and Psychology 8: 213–217

    Article  Google Scholar 

  • Wallach M.A., Wallach L. (1998b). When experiments serve purpose: misguided research in mainstream psychology. Theory and Psychology 8: 183–194

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Norwegian Knowledge Centre for the Health Services, Postbox 7004, St. Olavs Plass, N-130, Oslo, Norway

    Geir Smedslund

Authors
  1. Geir Smedslund
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geir Smedslund.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Smedslund, G. All Bachelors are Unmarried Men (p <  0.05). Qual Quant 42, 53–73 (2008). https://doi.org/10.1007/s11135-006-9036-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-006-9036-4

Keywords

Search

Navigation