Abstract
A preliminary theory of a temporary increase in the rate of an operant response with the transition to extinction (i.e., the extinction burst) is proposed. The theory assumes reinforcers are events permitting access to some valuable activity, and that such activity can compete for allocation with the target response under some conditions (e.g., very high reinforcement rates). With the transition to extinction, elimination of this competition for allocation can produce an increase in the the target response, but the increase is transient because the value of the target response decreases with exposure to extinction. The theory provides a way to understand why the extinction burst is not ubiquitous, seems more common following very small ratio schedules, occurs for a short period of time following the transition to extinction, and may be eliminated with the availability of alternative reinforcement. It appears to provide a reasonable starting point for a theory of the extinction burst that does not necessarily require inclusion of invigorating effects of frustration, and it is closely aligned with Resurgence as Choice theory. Additional research on factors modulating reinforcement-related activities and how they affect the extinction burst could help to further evaluate the theory.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
06 July 2022
A Correction to this paper has been published: https://doi.org/10.1007/s40614-022-00350-1
Notes
Reinforcement-access time may be easily identifiable and excluded from response-rate calculations when reinforcer deliveries are accompanied with a “time out” during which there is a period of access to a food hopper for pigeons or a liquid dipper for rats. However, when food pellets are delivered to rats, a quick perusal of the literature suggests that there is no standard convention in reporting whether or not a time-out period accompanies pellet deliveries. Thus, in many cases it is unclear whether or not any reinforcement access-time is being excluded from response rate calculations for rats receiving pellet reinforcers.
Thanks to Wayne Fisher for convincing me of this important point.
Again, the extent to which this already happens in studies with rats is presently unclear.
Of course, if a basic researcher is interested in running response-rates or the tempo of engaging in a behavior “while responding,” then reinforcement access time should be excluded from such calculations. If, however, one is interested in the overall ebb and flow of behavior across available activities in clock time as part of an environmental system that includes interacting with reinforcers (i.e., how does the organism spend its day), then it might make sense to include time spent engaging with reinforcers in response rate calculations.
If some smaller sample of experiences is chosen for n (e.g., Mazur, 1996), then n might rightfully be considered a free parameter. Note also that Shahan and Craig (2017) used what they called the scaled temporal weighting rule, which included a scaling exponent on t when t was measured in number of sessions. For the model of the extinction burst developed below, the scaling exponent has turned out to be unnecessary for the datasets to which it is applied below when t is measured in minutes, and so it has been omitted to reduce the number of parameters of the overall model.
This is a simplifying assumption because clearly the value associated with these two activities might not be exactly the same. For example, the value associated with reinforcement engagement (i.e., Vr) could often be greater because it involves immediate and direct contact with the reinforcer. Nevertheless, as shown below, the simplifying assumption that Vr =V works well enough for present purposes to demonstrate the potential utility of a matching-law based approach.
Of course, it is possible that some reinforcement-related activity (e.g., search of usual delivery site) might continue in the absence of reinforcer deliveries in some cases. A more complex model incorporating such potential lingering reinforcement-related behavior could be needed for datasets in the future, but as shown below the model does a good job without including such added complexity.
Note that previous applications of RaC have been conducted at the across-session level for which t in Equation 3 is measured in units of sessions, whereas the extinction burst model developed here to account for within-session increases in response rates used units of min for t.
It is noteworthy that even Skinner invoked frustration as an emotional effect playing a role in extinction. But when he did, such emotional effects were said to interfere with and reduce the operant response of interest (Skinner, 1938, 1950). For example, “When we fail to reinforce a response that has previously been reinforced, we not only initiate a process of extinction, we set up an emotional response—perhaps what is often meant by frustration. The pigeon coos in an identifiable pattern, moves rapidly about the cage, defecates, or flaps its wings rapidly in a squatting position that suggests treading (mating) behavior. This competes with the response of striking a key and is perhaps enough to account for the decline in rate in early extinction” (Skinner, 1950, pp. 203–204). The fact that one might employ frustration as either an invigorator or a suppressor of operant behavior during extinction highlights the unreasonable flexibility permitted by employing the concept in a purely informal, narrative account.
Neither does RaC nor the alternative potential momentum/MPR models described above.
References
Amsel, A. (1958). The role of frustrative nonreward in noncontinuous reward situations. Psychological Bulletin, 55(2), 102–119. https://doi.org/10.1037/h0043125
Amsel, A. (1992). Frustration theory: an analysis of dispositional learning and memory. Cambridge University Press. https://doi.org/10.1017/CBO9780511665561
Amsel, A., & Roussel, J. (1952). Motivational properties of frustration: I. Effect on a running response of the addition of frustration to the motivational complex. Journal of Experimental Psychology, 43(5), 363–368. https://doi.org/10.1037/h0059393.
Baum, W. M. (1981). Optimization and the matching law as accounts of instrumental behavior. Journal of the Experimental Analysis of Behavior, 36(3), 387–403. https://doi.org/10.1901/jeab.1981.36-387
Baum, W. M. (1993). Performances on ratio and interval schedules of reinforcement: data and theory. Journal of the Experimental Analysis of Behavior, 59(2), 245–264. https://doi.org/10.1901/jeab.1993.59-245.
Baum, W. M. (2012). Rethinking reinforcement: allocation, induction, and contingency. Journal of the Experimental Analysis of Behavior, 97(1), 101–124. https://doi.org/10.1901/jeab.2012.97-101
Baum, W. M., & Davison, M. (2014). Background activities, induction, and behavioral allocation in operant performance. Journal of the Experimental Analysis of Behavior, 102(2), 213–230. https://doi.org/10.1002/jeab.100
Baum, W. M., & Rachlin, H. C. (1969). Choice as time allocation. Journal of the Experimental Analysis of Behavior, 12(6), 861–874. https://doi.org/10.1901/jeab.1969.12-861
Boren, J. J. (1961). Resistance to extinction as a function of the fixed ratio. Journal of Experimental Psychology, 61(4), 304–308. https://doi.org/10.1037/h0040208
Carr, J. E., Bailey, J. S., Scott, C. L., Lucker, K. D., & Weil, T. M. (1998). On the effects of noncontingent delivery of differing magnitudes of reinforcement. Journal of Applied Behavior Analysis, 31, 313–321. https://doi.org/10.1901/jaba.1998.31-313
Coe, C. L., Stanton, M. E., & Levine, S. (1983). Adrenal responses to reinforcement and extinction: Role of expectancy versus instrumental responding. Behavioral Neuroscience, 97(4), 654–657. https://doi.org/10.1037/0735-7044.97.4.654
Craig, A. R., & Shahan, T. A. (2016). Behavioral momentum theory fails to account for the effects of reinforcement rate on resurgence. Journal of the Experimental Analysis of Behavior, 105(3), 375–392. https://doi.org/10.1002/jeab.207
Davison, M. C., & Hunter, I. W. (1979). Concurrent schedules: undermatching and control by previous experimental conditions. Journal of the Experimental Analysis of Behavior, 32(2), 233–244. https://doi.org/10.1901/jeab.1979.32-233
Davison, M., & McCarthy, D. (1988). The matching law: a research review. Lawrence Erlbaum Associates.
Davison, M., & Nevin, J. A. (1999). Stimuli, reinforcers, and behavior: an integration. Journal of the Experimental Analysis of Behavior, 71(3), 439–482. https://doi.org/10.1901/jeab.1999.71-439
de Villiers, P. A. (1977). Choice in concurrent schedules and a quantitative formulation of the law of effect. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 233–287). Prentice Hall.
de Villiers, P. A., & Herrnstein, R. J. (1976). Toward a law of response strength. Psychological Bulletin, 83(6), 1131–1153. https://doi.org/10.1037/0033-2909.83.6.1131
Devenport, L. D., & Devenport, J. A. (1994). Time-dependent averaging of foraging information in least chipmunks and golden-mantled ground squirrels. Animal Behaviour, 47(4), 787–802. https://doi.org/10.1006/anbe.1994.1111
Devenport, L., Hill, T., Wilson, M., & Ogden, E. (1997). Tracking and averaging in variable environments: a transition rule. Journal of Experimental Psychology: Animal Behavior Processes, 23(4), 450–460. https://doi.org/10.1037/0097-7403.23.4.450
Donny, E. C., Caggiula, A. R., Knopf, S., & Brown, C. (1995). Nicotine self-administration in rats. Psychopharmacology, 122(4), 390–394. https://doi.org/10.1007/bf02246272
Fisher, W. W., Thomspon, R. H., DeLeon, I. G., Piazza, C. C., Kuhn, D. E., Rodriguez-Catter, V., & Adelinis, J. D. (1999). Noncontingent reinforcement: effects of satiation versus choice responding. Research in Developmental Disabilities, 20(6), 411–427. https://doi.org/10.1016/s0891-4222(99)00022-0
Gibbon, J. (1995). Dynamics of time matching: arousal makes better seem worse. Psychonomic Bulletin & Review, 2(2), 208–215. https://doi.org/10.3758/BF03210960
Harris, A. C., Pentel, P. R., & Lesage, M. G. (2007). Prevalence, magnitude, and correlates of an extinction burst in drug-seeking behavior in rats trained to self-administer nicotine during unlimited access (23 h/day) sessions. Psychopharmacology, 194(3), 395–402. https://doi.org/10.1007/s00213-007-0848-2
Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4(3), 267–272. https://doi.org/10.1901/jeab.1961.4-267
Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13(2), 243–266. https://doi.org/10.1901/jeab.1970.13-243
Houston, A. I., Trimmer, P. C., & McNamara, J. M. (2021). Matching behaviours and rewards. Trends in Cognitive Sciences, 25(5), 403–415. https://doi.org/10.1016/j.tics.2021.01.011
Iigaya, K., Ahmadian, Y., Sugrue, L. P., Corrado, G. S., Loewenstein, Y., Newsome, W. T., & Fusi, S. (2019). Deviation from the matching law reflects an optimal strategy involving learning over multiple timescales. Nature Communications, 10(1), 1466. https://doi.org/10.1038/s41467-019-09388-3
Katz, B., & Lattal, K. A. (2020). An experimental analysis of the extinction-induced response burst. Journal of the Experimental Analysis of Behavior, 114(1), 24–46. https://doi.org/10.1002/jeab.611
Katz, B. R., & Lattal, K. A. (2021). What is an extinction burst? A case study in the analysis of transitional behavior. Journal of the Experimental Analysis of Behavior, 115(1), 129–140. https://doi.org/10.1002/jeab.642
Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology: a systematic text in the science of behavior. Appleton-Century-Crofts.
Killeen, P. R. (1994). Mathematical principles of reinforcement. Behavioral & Brain Sciences, 17(1), 105–172. https://doi.org/10.1017/S0140525X00033628
Killeen, P. R. (2014a). Pavlov + Skinner = Premack. International Journal of Comparative Psychology, 27(4), Article 24153.
Killeen, P. R. (2014b). A theory of behavioral contrast. Journal of the Experimental Analysis of Behavior, 102(3), 363–390. https://doi.org/10.1002/jeab.107
Killeen, P. R., & Jacobs, K. W. (2017). Coal is not black, snow is not white, food is not a reinforcer: the roles of affordances and dispositions in the analysis of behavior. The Behavior Analyst, 40(1), 17–38. https://doi.org/10.1007/s40614-016-0080-7
Killeen, P. R., & Nevin, J. A. (2018). The basis of behavioral momentum in the nonlinearity of strength. Journal of the Experimental Analysis of Behavior, 109(1), 4–32. https://doi.org/10.1002/jeab.304
Lattal, K. A. (1991). Scheduling positive reinforcers. In I. H. Iversen & K. A. Lattal (Eds.), Techniques in the behavioral and neural sciences, Vol. 6: Experimental analysis of behavior (pp. 87–134). Elsevier.
Lattal, K. A., Kuroda, T., & Cook, J. E. (2020). Early extinction effects following intermittent reinforcement: little evidence of extinction bursts. Journal of the Experimental Analysis of Behavior, 114(1), 47–59. https://doi.org/10.1002/jeab.616
Leitenberg, H., Rawson, R. A., & Mulick, J. A. (1975). Extinction and reinforcement of alternative behavior. Journal of Comparative and Physiological Psychology, 88(2), 640–652. https://doi.org/10.1037/h0076418
Lerman, D. C., & Iwata, B. A. (1995). Prevalence of the extinction burst and its attenuation during treatment. Journal of Applied Behavior Analysis, 28(1), 93–94. https://doi.org/10.1901/jaba.1995.28-93
Lerman, D. C., Iwata, B. A., & Wallace, M. D. (1999). Side effects of extinction: prevalence of bursting and aggression during the treatment of self-injurious behavior. Journal of Applied Behavior Analysis, 32(1), 1–8. https://doi.org/10.1901/jaba.1999.32-1
Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018). DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience, 21(9), 1281–1289. https://doi.org/10.1038/s41593-018-0209-y
Mazur, J. E. (1996). Past experience, recency, and spontaneous recovery in choice behavior. Animal Learning & Behavior, 24(1), 1–10. https://doi.org/10.3758/BF03198948
McDowell, J. J. (2019). On the current status of the evolutionary theory of behavior dynamics. Journal of the Experimental Analysis of Behavior, 111(1), 130–145. https://doi.org/10.1002/jeab.495
McDowell, J. J., & Kessel, R. (1979). A multivariate rate equation for variable-interval performance. Journal of the Experimental Analysis of Behavior, 31(2), 267–283. https://doi.org/10.1901/jeab.1979.31-267
Nevin, J. A., & Grace, R. C. (2000). Behavioral momentum and the law of effect. The Behavioral & Brain Sciences, 23(1), 73–130. https://doi.org/10.1017/s0140525x00002405
Nevin, J. A., & Grace, R. C. (2005). Resistance to extinction in the steady state and in transition. Journal of Experimental Psychology: Animal Behavior Processes, 31(2), 199–212. https://doi.org/10.1037/0097-7403.31.2.199
Nevin, J. A., Davison, M., & Shahan, T. A. (2005). A theory of attending and reinforcement in conditional discriminations. Journal of the Experimental Analysis of Behavior, 84(2), 281–303. https://doi.org/10.1901/jeab.2005.97-04
Nevin, J. A., Craig, A. R., Cunningham, P. J., Podlesnik, C. A., Shahan, T. A., & Sweeney, M. M. (2017). Quantitative models of persistence and relapse from the perspective of behavioral momentum theory: fits and misfits. Behavioural Processes, 141(Pt 1), 92–99. https://doi.org/10.1016/j.beproc.2017.04.016
Nist, A. N., & Shahan, T. A. (2021). The extinction burst: impact of reinforcement time and level of analysis on measured prevalence. Journal of the Experimental Analysis of Behavior, 116, 131–148. https://doi.org/10.1002/jeab.714
Niyuhire, F., Varvel, S. A., Thorpe, A. J., Stokes, R. J., Wiley, J. L., & Lichtman, A. H. (2007). The disruptive effects of the CB1 receptor antagonist rimonabant on extinction learning in mice are task-specific. Psychopharmacology, 191(2), 223–231. https://doi.org/10.1007/s00213-006-0650-6
Papini, M. R. (2003). Comparative psychology of surprising nonreward. Brain, Behavior & Evolution, 62(2), 83–95. https://doi.org/10.1159/000072439
Papini, M. R., & Dudley, R. T. (1997). Consequences of surprising reward omissions. Review of General Psychology, 1(2), 175–197. https://doi.org/10.1037/1089-2680.1.2.175
Pushparaj, A., Pryslawsky, Y., Forget, B., Yan, Y., & Le Foll, B. (2012). Extinction bursts in rats trained to self-administer nicotine or food in 1-h daily sessions. American Journal of Translational Research, 4(4), 422–431.
Roscoe, E. M., Iwata, B. A., & Rand, M. S. (2003). Effects of reinforcer consumption and magnitude on response rates during noncontingent reinforcement. Journal of Applied Behavior Analysis, 36(4), 525–539. https://doi.org/10.1901/jaba.2003.36-525
Salamone, J. D., Kurth, P., McCullough, L. D., & Sokolowski, J. D. (1995). The effects of nucleus accumbens dopamine depletions on continuously reinforced operant responding: contrasts with the effects of extinction. Pharmacology, Biochemistry, & Behavior, 50(3), 437–443. https://doi.org/10.1016/0091-3057(94)00294-s
Schramm-Sapyta, N. L., Olsen, C. M., & Winder, D. G. (2006). Cocaine self-administration reduces excitatory responses in the mouse nucleus accumbens shell. Neuropsychopharmacology, 31(7), 1444–1451. https://doi.org/10.1038/sj.npp.1300918
Shahan, T. A. (2017). Moving beyond reinforcement and response strength. The Behavior Analyst, 40(1), 107–121. https://doi.org/10.1007/s40614-017-0092-y
Shahan, T. A., & Craig, A. R. (2017). Resurgence as choice. Behavioural Processes, 141, 100–127. https://doi.org/10.1016/j.beproc.2016.10.006
Shahan, T. A., & Sweeney, M. M. (2011). A model of resurgence based on behavioral momentum theory. Journal of the Experimental Analysis of Behavior, 95(1), 91–108. https://doi.org/10.1901/jeab.2011.95-91
Shahan, T. A., Browning, K. O., & Nall, R. W. (2020a). Resurgence as choice in context: treatment duration and on/off alternative reinforcement. Journal of the Experimental Analysis of Behavior, 113(1), 57–76. https://doi.org/10.1002/jeab.563
Shahan, T. A., Browning, K. O., Nist, A. N., & Sutton, G. M. (2020b). Resurgence and downshifts in alternative reinforcement rate. Journal of the Experimental Analysis of Behavior, 114, 163–178. https://doi.org/10.1002/jeab.625
Skinner, B. F. (1938). The behavior of organisms: an experimental analysis. Appleton-Century.
Skinner, B. F. (1950). Are theories of learning necessary? Psychological Review, 57(4), 193–216. https://doi.org/10.1037/h0054367
Skinner, B. F. (1979). The shaping of a behaviorist. Alfred A. Knopf.
Stout, S. C., Muzio, R. N., Boughner, R. L., & Papini, M. R. (2002). Aftereffects of the surprising presentation and omission of appetitive reinforcers on key-pecking performance in pigeons. Journal of Experimental Psychology: Animal Behavior Processes, 28(3), 242–256. https://doi.org/10.1037/0097-7403.28.3.242
Stout, S. C., Boughner, R. L., & Papini, M. R. (2003). Reexamining the frustration effect in rats: aftereffects of surprising reinforcement and nonreinforcement. Learning & Motivation, 34(4), 437–456. https://doi.org/10.1016/S0023-9690(03)00038-9
Stubbs, D. A., & Pliskoff, S. S. (1969). Concurrent responding with fixed relative rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12(6), 887–895. https://doi.org/10.1901/jeab.1969.12-887
Sweeney, M. M., & Shahan, T. A. (2013). Effects of high, low, and thinning rates of alternative reinforcement on response elimination and resurgence. Journal of the Experimental Analysis of Behavior, 100(1), 102–116. https://doi.org/10.1002/jeab.26
Vollmer, T. R., Iwata, B. A., Zarcone, J. R., Smith, R. G., & Mazaleski, J. L. (1993). The role of attention in the treatment of attention-maintained self-injurious behavior: noncontingent reinforcement and differential reinforcement of other behavior. Journal of Applied Behavior Analysis, 26(1), 9–21. https://doi.org/10.1901/jaba.1993.26-9
Ward, S. J., Rosenberg, M., Dykstra, L. A., & Walker, E. A. (2009). The CB1 antagonist rimonabant (SR141716) blocks cue-induced reinstatement of cocaine seeking and other context and extinction phenomena predictive of relapse. Drug & Alcohol Dependence, 105(3), 248–255. https://doi.org/10.1016/j.drugalcdep.2009.07.002
Winterrowd, M. F., & Devenport, L. D. (2004). Balancing variable patch quality with predation risk. Behavioural Processes, 67(1), 39–46. https://doi.org/10.1016/j.beproc.2004.02.003
Acknowledgments
Thanks to Brian Greer and Tom Critchfield for comments on a previous version, Wayne Fisher for discussions on the topic, and Tony Nist and Jack Van Allsburg for their contributions.
Funding
This work was funded by grant R01HD093734 (TAS) from the Eunice K. Shriver National Institute of Child Health and Human Development.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
The author has no known conflicts of interest to report.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article has been updated to provide correct figure characters in Figure 2.
Rights and permissions
About this article
Cite this article
Shahan, T.A. A Theory of the Extinction Burst. Perspect Behav Sci 45, 495–519 (2022). https://doi.org/10.1007/s40614-022-00340-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40614-022-00340-3