Skip to main content
SpringerLink
Log in

A Critique of the Usefulness of Inferential Statistics in Applied Behavior Analysis

  • Published:
The Behavior Analyst Aims and scope Submit manuscript

Abstract

Researchers continue to recommend that applied behavior analysts use inferential statistics in making decisions about effects of independent variables on dependent variables. In many other approaches to behavioral science, inferential statistics are the primary means for deciding the importance of effects. Several possible uses of inferential statistics are considered. Rather than being an objective means for making decisions about effects, as is often claimed, inferential statistics are shown to be subjective. It is argued that the use of inferential statistics adds nothing to the complex and admittedly subjective nonstatistical methods that are often employed in applied behavior analysis. Attacks on inferential statistics that are being made, perhaps with increasing frequency, by those who are not behavior analysts, are discussed. These attackers are calling for banning the use of inferential statistics in research publications and commonly recommend that behavioral scientists should switch to using statistics aimed at interval estimation or the method of confidence intervals. Interval estimation is shown to be contrary to the fundamental assumption of behavior analysis that only individuals behave. It is recommended that authors who wish to publish the results of inferential statistics be asked to justify them as a means for helping us to identify any ways in which they may be useful.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abelson, R. P. (1997). On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12–15.

    Article  Google Scholar 

  • Atkinson, D. R., Furlong, M. J., & Wampold, B. E. (1982). Statistical significance, reviewer evaluations, and the scientific process: Is there a (statistically) significant relationship? Journal of Counseling Psychology, 29, 189–194.

    Article  Google Scholar 

  • Baer, D. M. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10, 167–172.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bakan, D. (1966). The test of significance in psychological research. Psychological Research, 66, 423–437.

    Google Scholar 

  • Barlow, D. H. (Ed.). (1983). Journal of Applied Behavior Analysis, 16(2, 3).

  • Bernard, C. (1927). An introduction to the study of experimental medicine. New York: Macmillan. (Original work published 1865)

    Google Scholar 

  • Branch, M. N. (Ed.). (1993). Journal of the Experimental Analysis of Behavior, 60.

  • Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.

    Article  Google Scholar 

  • Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49, 997–1003.

    Article  Google Scholar 

  • Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus, OH: Merrill.

    Google Scholar 

  • Costa, L., & Rourke, B. P. (Eds.). (1994). Journal of Clinical and Experimental Neuropsychology, 16(1).

  • DeRouen, T. A. (1987). Comment on statistical testing and confidence intervals. American Journal of Public Health, 77, 237.

    Article  Google Scholar 

  • Doob, L. W. (Ed.). (1994). Journal of Social Psychology, 134(4).

  • Dugbartey, A. T. (1994). The factor structure of traditional beliefs among Ghanian university students. Journal of Social Psychology, 134, 549–550.

    Article  Google Scholar 

  • Edgington, E. S. (1980). Random assignment and statistical tests for one-subject experiments. Behavioral Assessment, 2, 19–28.

    Google Scholar 

  • Edgington, E. S. (1982). Non-parametric tests for single-subject multiple schedule experiments. Behavioral Assessment, 4, 83–91.

    Google Scholar 

  • Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh, Scotland: Oliver and Boyd.

    Google Scholar 

  • Fleiss, J. L. (1986). Significance tests have a role in epidemiologic research: Reactions to A. M. Walker. American Journal of Public Health, 76, 559–560.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gentile, J. R., Roden, A. H., & Klein, R. D. (1972). An analysis of variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5, 193–198.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gigerenzer, G., & Murray, D. J. (1987). Cognition as intuitive statistics. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Gottman, J. M. (1981). Time-series analysis: A comprehensive introduction for social scientists. Cambridge, England: Cambridge University Press.

    Google Scholar 

  • Gottman, J. M., & Glass, G. V. (1978). Analysis of interrupted time-series experiments. In T. R. Kratochwill (Ed.), Single-subject research: Strategies for evaluating change (pp. 197–235). New York: Academic Press.

    Chapter  Google Scholar 

  • Hartmann, D. P. (1974). Forcing square pegs into round holes: Some comments on “An analysis of variance model for the intrasubject replication design.” Journal of Applied Behavior Analysis, 7, 635–638.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hartmann, D. P., Gottman, J. M., Jones, R. R., Gardner, W., Kazdin, A. E., & Vaught, R. (1980). Interrupted time-series analysis and its application to behavioral data. Journal of Applied Behavior Analysis, 13, 543–559.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hays, W. L. (1963). Statistics for psychologists. New York: Holt, Rinehart, and Winston.

    Google Scholar 

  • Hogben, L. (1957). Statistical theory: The relationship of probability, credibility, and error: An examination of the contemporary crisis in statistical theory from a behaviourist viewpoint. London: Allen and Unwin.

    Google Scholar 

  • Horne, G. P., Yang, M. C. K., & Ware, W. B. (1982). Time-series analysis for single-subject designs. Psychological Bulletin, 91, 178–189.

    Article  Google Scholar 

  • Howell, D. C. (1992). Statistical methods for psychology (3rd ed.). Boston: PWS-Kent.

    Google Scholar 

  • Huitema, B. E. (1986). Statistical analysis and single-subject designs. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis (pp. 209–232). New York: Plenum.

    Chapter  Google Scholar 

  • Hulse, S. H. (Ed.). (1994). Journal of Experimental Psychology: Animal Behavior Processes, 20(3).

  • Hunt, E. (Ed.). (1994). Journal of Experimental Psychology: General, 123(2).

  • Hunter, J. E. (1997). Needed: A ban on the significance test. Psychological Science, 8, 3–7.

    Article  Google Scholar 

  • Hurlburt, R. T. (1994). Comprehending behavioral statistics. Pacific Grove, CA: Brooks/Cole.

    Google Scholar 

  • Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and tactics of behavioral research (2nd ed.). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Jones, R. R., Vaught, R. S., & Weinrott, M. (1977). Time-series analysis in operant research. Journal of Applied Behavior Analysis, 10, 151–166.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kazdin, A. E. (1976). Statistical analyses for single-case experimental designs. In M. Hersen & D. H. Barlow (Eds.), Single-case experimental designs: Strategies for studying behavior change (pp. 265–316). Oxford, England: Pergamon Press.

    Google Scholar 

  • Keselman, H. J., & Leventhal, L. (1974). Concerning the statistical procedures enumerated by Gentile et al.: Another perspective. Journal of Applied Behavior Analysis, 7, 643–645.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kratochwill, T. (Ed.). (1974). Single-subject research: Strategies for evaluating change. New York: Academic Press.

    Google Scholar 

  • Kratochwill, T., Alden, K., Demuth, D., Dawson, D., Panicucci, C., Arnston, P., McMurray, N., Hemstead, J., & Levin, J. (1974). A further consideration in the application of an analysis of variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 7, 629–633.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kratochwill, T., & Levin, J. R. (1980). On the applicability of various data analysis procedures to the simultaneous and alternating treatment designs in behavior therapy research. Behavioral Assessment, 2, 353–360.

    Google Scholar 

  • Lachenbruch, P. A., Clark, V. A., Cumberland, W. G., Chang, P. C., Afifi, A. A., Flack, V. F., & Elashoff, R. M. (1987). Comment on statistical testing and confidence intervals. American Journal of Public Health, 77, 237.

    Article  PubMed Central  Google Scholar 

  • Levin, J. R. (Ed.). (1994). Journal of Educational Psychology, 86(3).

  • Mainstone, L. E., & Levi, A. S. (1987). Fundamentals of statistical process control. Journal of Organizational Behavior Management, 9, 5–21.

    Article  Google Scholar 

  • Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806–834.

    Article  Google Scholar 

  • Michael, J. (1974). Statistical inference for individual organism research: Mixed blessing or curse? Journal of Applied Behavior Analysis, 7, 647–653.

    Article  PubMed  PubMed Central  Google Scholar 

  • Neef, N. A. (Ed.). (1993). Journal of Applied Behavior Analysis, 26(2, 3).

  • Nevin, J. A. (Ed.). (1983). Journal of the Experimental Analysis of Behavior, 40.

  • Notz, W. W., Boschman, I., & Tax, S. T. (1987). Reinforcing punishment and extinguishing reward: On the folly of OBM without SPC. Journal of Organizational Behavior Management, 9, 33–46

    Article  Google Scholar 

  • Parsonson, B. S., & Baer, D. M. (1986). The graphic analysis of data. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis (pp. 157–186). New York: Plenum.

    Chapter  Google Scholar 

  • Pfadt, A., Cohen, I. L., Sudhalter, V., Romanczyk, R. G., & Wheeler, D. J. (1992). Applying statistical process control to clinical data: An illustration. Journal of Applied Behavior Analysis, 25, 551–560.

    Article  PubMed  PubMed Central  Google Scholar 

  • Pfadt, A., & Wheeler, D. J. (1995). Using statistical process control to make data-based clinical decisions. Journal of Applied Behavior Analysis, 28, 349–370.

    Article  PubMed  PubMed Central  Google Scholar 

  • Poole, C. (1987). Beyond the confidence interval. American Journal of Public Health, 77, 195–199.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rayner, K. (Ed.). (1994). Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(5).

  • Rosnow, R. L., & Rosenthal, R. (1988). Focused tests of significance and effect size estimation in counseling psychology. Journal of Counseling Psychology, 35, 203–208.

    Article  Google Scholar 

  • Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276–1284.

    Article  Google Scholar 

  • Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance test. Psychological Bulletin, 57, 416–428.

    Article  PubMed  Google Scholar 

  • Savitz, D. (1987). Comment on statistical testing and confidence intervals. American Journal of Public Health, 77, 237–238.

    Article  Google Scholar 

  • Schmitt, N. (Ed.). (1994). Journal of Applied Psychology, 79(3).

  • Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309–316.

    Article  Google Scholar 

  • Shrot, P. E. (1997). Should significance tests be banned? Introduction to a special section exploring the pros and cons. Psychological Science, 8, 1–2.

    Article  Google Scholar 

  • Sidman, M. (1960). Tactics of scientific research. New York: Basic Books.

    Google Scholar 

  • Somerville, S. C. (Ed.). (1994). Child Development 65(3).

  • Spence, J. T., Cotton, J. W., Underwood, B. J., & Duncan, C. P. (1990). Elementary statistics (5th ed.). Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Thompson, W. D. (1987). Statistical criteria in the interpretation of epidemiologic data. American Journal of Public Health, 77, 191–194.

    Article  PubMed  PubMed Central  Google Scholar 

  • Thoresen, C. E., & Elashoff, J. D. (1974). “An analysis-of-variance model for intrasubject replication design”: Some additional comments. Journal of Applied Behavior Analysis, 7, 639–641.

    Article  PubMed  PubMed Central  Google Scholar 

  • Van Meter, P., Yokoi, L., & Pressley, M. (1994). College students’ theory of note taking derived from their perceptions of note-taking. Journal of Educational Psychology, 86, 323–338.

    Article  Google Scholar 

  • Walker, A. M. (1986). Reporting the results of epidemiologic studies. American Journal of Public Health, 76, 556–558.

    Article  PubMed  PubMed Central  Google Scholar 

  • Williams, D. G. (1994). Population density and mental illness. Journal of Social Psychology, 134, 545–546.

    Article  PubMed  Google Scholar 

  • Wolff, A. B., Sass, K. J., & Keidan, J. (1994). Case report of an intracarotid amobarbital procedure performed for a deaf patient. Journal of Clinical and Experimental Neuropsychology, 16, 15–20.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Auburn University, Auburn, Alabama, 36849, USA

    B. L. Hopkins, Brian L. Cole & Tina L. Mason

Authors
  1. B. L. Hopkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brian L. Cole
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tina L. Mason
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

The initial draft of this manuscript was written while the first author was on a sabbatical hosted by the Department of Psychology at Emory University.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hopkins, B.L., Cole, B.L. & Mason, T.L. A Critique of the Usefulness of Inferential Statistics in Applied Behavior Analysis. BEHAV ANALYST 21, 125–137 (1998). https://doi.org/10.1007/BF03392787

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03392787

Key words

Search

Navigation