Abstract
Behavior analysts make dozens of practice-related decisions every day. Past research has extensively examined practice-related decision making by medical and other healthcare professionals, but decision making by practicing behavior analysts has garnered little research attention. The purpose of this proof of concept study was to begin a translational research agenda toward understanding what variables influence behavior analyst decision making and precisely how those variables do so. Behavior analyst decision making often involves data and the most fundamental characteristics of data are those indicating data trustworthiness—validity, reliability, and accuracy. To isolate a single independent variable, we used a lengthening data path procedure to parametrically assess how reducing data accuracy changed decisions to continue or modify an intervention in 30 students of behavior-analytic masters or doctoral programs. When data accuracy was 100%, most participants waited 9–10 trials before intervening. When data accuracy was below 60%, most participants waited 4–6 trials before intervening. To begin exploring potential behavioral processes underlying the influence of data accuracy on practice-related decisions, we also examined how well one popular description of decision making described participant choice—probability discounting. Probability discounting described the pattern of choices well for 16 of the 30 participants, suggesting other analytic frameworks should be explored. Nevertheless, data accuracy systematically influenced the majority of participants choices to continue or modify an intervention, although the degree of influence differed on an individual basis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Asgarova, R., Macaskill, A. C., & Hunt, M. J. (2020). Gain-loss asymmetry in experiential probability discounting. The Psychological Record, 70, 359–371. https://doi.org/10.1007/s40732-020-00379-1.
Baum, W. M. (2018). Three laws of behavior: Allocation, induction, and covariance. Behavior Analysis: Research & Practice, 18, 239–251. https://doi.org/10.1037/bar0000104.
Blair, B. J., Tarbox, J., Albright, L., MacDonald, J. M., Shawler, L. A., Russo, S. R., & Dorsey, M. F. (2019). Using equivalence-based instruction to teach visual analysis of graphs. Behavioral Interventions, 34, 405–418. https://doi.org/10.1002/bin.1669.
Bourret, J. C., & Pietras, C. J. (2013). Visual analysis in single-case research. In G. J. Madden, W. V. Dube, & K. A. Lattal (Eds.), APA handbook of behavior analysis, Vol. 1: Methods and principles (pp. 199–218). Washington, DC: American Psychological Association.
Boykin, R. A., & Nelson, R. O. (1981). The effects of instructions and calculation procedures on observers’ accuracy, agreement, and calculation correctness. Journal of Applied Behavior Analysis, 14, 479–489.
Bruce, J. M., Bruce, A. S., Catley, D., Lynch, S., Goggin, K., & Jarmolowicz, D. P. (2016). Being kind to your future self: Probability discounting of health decision-making. Annals of Behavioral Medicine, 50(2), 297–309. https://doi.org/10.1007/s12160-015-9754-8.
Carroll, R. A., Kodak, T., & Fisher, W. W. (2013). An evaluation of programmed treatment-integrity errors during discrete-trial instruction. Journal of Applied Behavior Analysis, 46, 379–394. https://doi.org/10.1002/jaba.49.
Cooper, J.O., Heron, T.E., & Heward, W.L. (2007). Applied Behavior Analysis: Second Edition. New York, NY: Pearson.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied Behavior Analysis: Third Edition. New York, NY: Pearson.
Cox, D. J., & Dallery, J. (2018). Verbal behavior and risky choice in humans: Exploring the boundaries of the description-experience gap. Behavioural Processes, 157, 301–308. https://doi.org/10.1016/j.beproc.2018.09.002.
Critchfield, T. S. (2011). Translational contributions of the experimental analysis of behavior. The Behavior Analyst, 34, 3–17.
Dallery, J., & Soto, P. L. (2013). Quantitative description of environment-behavior relations. In G. J. Madden, W. V. Dube, & K. A. Lattal (Eds.), APA handbook of behavior analysis, Vol. 1: Methods and principles (pp. 219–249). Washington, DC: American Psychological Association.
Dart, E. H., & Radley, K. C. (2017). The impact of ordinate scaling on the visual analysis of single-case data. Journal of School Psychology, 63, 105–118. https://doi.org/10.1016/j.jsp.2017.03.008.
DeHart, W. B., Friedel, J. E., Frye, C. C. J., Galizio, A., & Odum, A. L. (2018). The effects of outcome unit framing on delay discounting. Journal of the Experimental Analysis of Behavior, 110, 412–429. https://doi.org/10.1002/jeab.469.
DeHart, W. B., & Odum, A. L. (2015). The effects of the framing of time on delay discounting. Journal of the Experimental Analysis of Behavior, 103, 10–21. https://doi.org/10.1002/jeab.125.
DeProspero, A., & Cohen, S. (1979). Inconsistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 12, 573–579. https://doi.org/10.1901/jaba.1979.12-573.
Diller, J. W., Barry, R. J., & Gelino, B. W. (2016). Visual analysis of data in a multielement design. Journal of Applied Behavior Analysis, 49, 980–985. https://doi.org/10.1002/jaba.325.
Du, W., Green, L., & Myerson, J. (2002). Cross-cultural comparisons of discounting delayed and probabilistic rewards. Psychological Record, 52, 479–492.
Estes, W. K. (1976). The cognitive side of probability learning. Psychological Review, 83, 37–64. https://doi.org/10.1037/0033-295X.83.1.37.
Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. Journal of Applied Behavior Analysis, 36, 387–406. https://doi.org/10.1901/jaba.2003.36-387.
Fryling, M. J., Wallace, M. D., & Yassine, J. N. (2012). Impact of treatment integrity on intervention effectiveness. Journal of Applied Behavior Analysis, 45, 449–453. https://doi.org/10.1901/jaba.2012.45-449.
Gallistel, C. R. (2012). Extinction from a rationalist perspective. Behavioural Processes, 90(1), 66–80. https://doi.org/10.1016/j.beproc.2012.02.008.
Geiger, K. B., Carr, J. E., & LeBlanc, L. A. (2010). Function-based treatments for escape-maintained problem behavior: A treatment-selection model for practicing behavior analysts. Behavior Analysis in Practice, 3(1), 22–32. https://doi.org/10.1007/BF03391755.
Green, L., Myerson, J., & Ostaszewski, P. (1999). Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes. Journal of Experimental Psychology: Learning, Memory, & Cognition, 25, 418–427.
Hayes, S. C. (1989). Rule-governed behavior: Cognition, contingencies & instructional control (reprint ed.). Reno, NV: Context Press. https://psycnet.apa.org/record/1989-98580-000.
Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267–272. https://doi.org/10.1901/jeab.1961.4-267.
Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13, 243–266. https://doi.org/10.1901/jeab.1970.13-243.
Hinz, K. L., McGee, H. M., Huitema, B. E., Dickinson, A. M., & Van Enk, R. A. (2014). Observer accuracy and behavior analysis: data collection procedures on hand hygiene compliance in a neurovascular unit. American Journal of Infection Control, 42, 1067–1073.
Hursh, S. R., & Silberberg, A. (2008). Economic demand and essential value. Psychological Review, 115, 186–198.
Jarmolowicz, D. P., Reed, D. D., Bruce, A. S., & Bruce, J. (2019). On the behavioral economics of medication: A research story. Behavioural Processes, 165, 66–77. https://doi.org/10.1016/j.beproc.2019.05.019.
Jarmolowicz, D. P., Reed, D. D., Bruce, A. S., Catley, D., Lynch, S., Goggin, K., … Bruce, J. M. (2016). Using EP50 to forecast treatment adherence in individuals with multiple sclerosis. Behavioural Processes, 132, 94–99. https://doi.org/10.1016/j.beproc.2016.09.003
Johnson, M. W., Herrmann, E. S., Sweeney, M. M., LeComte, R. S., & Johnson, P. S. (2017). Cocaine administration dose-dependently increases sexual desire and decreases condom use likelihood: The role of delay and probability discounting in connecting cocaine with HIV. Psychopharmacology, 234(4), 599–612. https://doi.org/10.1007/s00213-016-4493-5.
Johnson, P. S., Sweeney, M. M., Herrmann, E. S., & Johnson, M. W. (2016). Alcohol increases delay and probability discounting of condom-protected sex: A novel vector for alcohol-related HIV transmission. Alcoholism: Clinical & Experimental Research, 40(6), 1339–1350. https://doi.org/10.1111/acer.13079.
Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and tactics of behavioral research (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Kahng, S. W., Chung, K. M., Gutshall, K., Pitts, S. C., Kao, J., & Girolami, K. (2010). Consistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 43, 35–45. https://doi.org/10.1901/jaba.2010.43-35.
Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). New York, NY: Oxford University Press.
Kipfmiller, K. J., Brodhead, M. T., Wolfe, K., LaLonde, K., Sipila, E. S., Bak, S. M. Y., & Fisher, M. H. (2019). Training front-line employees to conduct visual analysis using a clinical decision-making model. Journal of Behavioral Education. https://doi.org/10.1007/s10864-018-09318-1.
Kirby, K. N., & Marakovic, N. N. (1996). Delay-discounting probabilistic rewards: Rates decrease as amounts increase. Psychonomic Bulletin & Review, 3, 100–104.
Ledford, J. R., & Gast, D. L. (2018). Single case research methodology (3rd ed.). New York, NY: Routledge.
Lerman, D. C., Hovanetz, A., Strobel, M., & Tetreault, A. (2009). Accuracy of teacher-collected descriptive analysis data: a comparison of narrative and structured reading formats. Journal of Behavioral Education, 18, 157–172.
Mace, F. C., & Critchfield, T. S. (2010). Translational research in behavior analysis: Historical traditions and imperative for the future. Journal of the Experimental Analysis of Behavior, 93, 293–312. https://doi.org/10.1901/jeab.2010.93-293.
Maffei-Almodovar, L., Feliciano, G., Fienup, D. M., & Sturmey, P. (2017). The use of behavioral skills training to teach graph analysis to community based teachers. Behavior Analysis in Practice, 10, 355–362. https://doi.org/10.1007/s40617-017-0199-3.
McKerchar, T. L., Green, L., & Myerson, J. (2010). On the scaling interpretation of exponents in hyperboloid models of delay and probability discounting. Behavioural Processes, 84, 440–444. https://doi.org/10.1016/j.beproc.2010.01.003.
McKerchar, T. L., & Renda, R. C. (2012). Delay and probability discounting in humans: An overview. The Psychological Record, 62, 817–834.
Mudford, O. C., Martin, N. M., Hui, J. K. Y., & Taylor, S. A. (2009). Assessing observer accuracy in continuous recording of rate and duration: three algorithms compared. Journal of Applied Behavior Analysis, 42, 527–539. https://doi.org/10.1901/jaba.2009.42-527.
Naude, G. P., Kaplan, B. A., Reed, D. D., Henley, A. J., & DiGennaro Reed, F. D. (2018). Temporal framing and the hidden-zero effect: Rate-dependent outcomes on delay discounting. Journal of the Experimental Analysis of Behavior, 109, 506–519. https://doi.org/10.1002/jeab.328.
Ninci, J., Vannest, K. J., Willson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single-case data: A meta-analysis. Behavior Modification, 39, 510–541. https://doi.org/10.1177/0145445515581327.
O’Leary, K. D., Kent, R. N., & Kanowitz, J. (1975). Shaping data collection congruent with experimental hypotheses. Journal of Applied Behavior Analysis, 8, 43–51. https://doi.org/10.1901/jaba.1975.8-43.
Pence, S. T., & St. Peter, C. C. (2015). Evaluation of treatment integrity errors on mand acquisition. Journal of Applied Behavior Analysis, 48, 575–589. https://doi.org/10.1002/jaba.238.
Poling, A. (2010). Looking to the future: Will behavior analysis survive and prosper? The Behavior Analyst, 33, 7–17. https://doi.org/10.1007/bf03392200.
Qualtrics. (2019). Qualtrics (software; version June, 2019).. Provo, UT: Author. Retrieved from https://www.qualtrics.com
Rachlin, H., Brown, J., & Cross, D. (2000). Discounting in judgments of delay and probability. Journal of Behavioral Decision Making, 13, 145–159.
Rachlin, H., & Green, L. (1972). Commitment, choice and self-control. Journal of the Experimental Analysis of Behavior, 17, 15–22. https://doi.org/10.1901/jeab.1972.17-15.
Rachlin, H., Raineri, A., & Cross, D. (1991). Subjective probability and delay. Journal of the Experimental Analysis of Behavior, 55, 233–244.
Rasmussen, E. B., Lawyer, S. R., & Reilly, W. (2010). Percent body fat is related to delay and probability discounting for food in humans. Behavioural Processes, 83, 23–30. https://doi.org/10.1016/j.beproc.2009.09.001.
Repp, A. C., Nieminen, G. S., Olinger, E., & Brusca, R. (1988). Direct observation: Factors affecting the accuracy of observers. Exceptional Children, 55, 29–36. https://doi.org/10.1177/001440298805500103.
Rescorla, R. A. (1967). Pavlovian conditioning and its proper control procedures. Psychological Review, 74, 71–80. https://doi.org/10.1037/h0024109.
Rescorla, R. A. (1968). Pavlovian conditioning. It’s not what you think it is. American Psychologist, 43, 151–160. https://doi.org/10.1037/003-066X.43.3.151.
Saini, V., Fisher, W. W., & Retzlaff, B. J. (2018). Predictive validity and efficiency of ongoing visual-inspection criteria for interpreting functional analyses. Journal of Applied Behavior Analysis, 51, 303–320. https://doi.org/10.1002/jaba.450.
Seabold, S., & Perktold, J. (2010). statsmodels: Econometric and statistical modeling with python. Proceedings of the 9th Python in Science Conference. Retrieved from https://www.statsmodels.org/stable/index.html.
Shead, N. W., & Hodgins, D. C. (2009). Probability discounting of gains and losses: Implications for risk attitudes and impulsivity. Journal of the Experimental Analysis of Behavior, 92, 1–16. https://doi.org/10.1901/jeab.2009.92-1.
Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. New York, NY: Basic Books.
St. Peter, C. C., Byrd, J. D., Pence, S. T., & Foreman, A. P. (2016). Effects of treatment-integrity failures on a response-cost procedure. Journal of Applied Behavior Analysis, 49, 308–328. https://doi.org/10.1002/jaba.291.
Staddon, J. E. R., Hinson, J. M., & Kram, R. (1981). Optimal choice. Journal of the Experimental Analysis of Behavior, 35, 397–412.
Vanselow, N. R., Thompson, R., & Karsina, A. (2011). Data-based decision making: The impact of data variability, training, and context. Journal of Applied Behavior Analysis, 44, 767–780. https://doi.org/10.1901/jaba.2011.44-767.
Ward, R. D., Gallistel, C. R., & Balsam, P. D. (2013). It’s the information. Behavioural Processes, 95, 3–7. https://doi.org/10.1016/j.beproc.2013.01.005.
Wolfe, K., Barton, E., & Meadan, H. (2019). Systematic protocols for the visual analysis of single-case research data. Behavior Analysis in Practice, 12, 491–502. https://doi.org/10.1007/s40617-019-00336-7.
Wolfe, K., & McCammon, M. N. (in press). The analysis of single-case research data: current instructional practices. Journal of Behavioral Education. Advance online publication. https://doi.org/10.1007/s10864-020-09403-4.
Wolfe, K., & Slocum, T. A. (2015). A comparison of two approaches to training visual analysis of AB graphs. Journal of Applied Behavior Analysis, 48, 472–477. https://doi.org/10.1002/jaba.212.
Wulff, D. U., Mergenthaler-Canseco, M., & Hertwig, R. (2018). A meta-analytic review of two modes of learning and the description-experience gap. Psychological Bulletin, 144(2), 140–176. https://doi.org/10.1037/bul0000115.
Yi, R., & Bickel, W. K. (2005). Representation of odds against in terms of frequencies reduces probability discounting. The Psychological Record, 55, 577–593.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
The researchers do not have any potential conflicts of interest to report. All research procedures were performed in accordance with the 1964 Helsinki declaration, its later amendments or comparable ethical standards, and informed consent was obtained from all individual participants included in the study.
Data Accessibility
Raw data can be accessed at the following repository: https://github.com/davidjcox333/Data-Accuracy-ABA-Decision-Making
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Cox, D.J., Brodhead, M.T. A Proof of Concept Analysis of Decision-Making with Time-Series Data. Psychol Rec 71, 349–366 (2021). https://doi.org/10.1007/s40732-020-00451-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40732-020-00451-w