Elsevier

Learning and Motivation

Volume 68, November 2019, 101587
Learning and Motivation

Renewal in a heterogeneous behavior chain: Extinction of the first response prevents renewal of a second response when it is separately extinguished and returned to the chain

https://doi.org/10.1016/j.lmot.2019.101587Get rights and content

Abstract

Two experiments with rat subjects examined the renewal of an extinguished instrumental response that occurs when it is returned to the context of a behavior chain in which it had been trained. In both experiments, rats first learned a discriminated heterogeneous chain in which a stimulus (S1) set the occasion for one response (R1), the emission of which turned off S1 and turned on a second stimulus (S2) that set the occasion for a second response (R2). R2 in turn terminated S2 and earned a food-pellet reinforcer. When R2 was extinguished separate from the chain, it was renewed when it was returned to and tested in the chain. However, in both experiments, separate extinction of R1 prevented this renewal of R2 from occurring. In Experiment 2, Pavlovian extinction of S1 without the opportunity to emit R1 during extinction also weakened the renewal of R2 and had an unexpected effect of weakening R1. The results are consistent with the idea that R1 can be an important part of the “context” controlling R2 in a discriminated chain. They also suggest that Pavlovian extinction of a discriminative stimulus can weaken the instrumental response under some conditions.

Introduction

Recent research from this laboratory has studied the associative structure underlying discriminated heterogeneous behavior chains (see Thrailkill & Bouton, 2016a, 2017 for reviews). In such chains, a stimulus (S1) sets the occasion for an instrumental “procurement” response (R1). Completion of R1 during S1 terminates S1 and turns on a second stimulus (S2), which now sets the occasion for a second instrumental “consumption” response (R2). Completion of R2 turns off S2 and delivers a primary reinforcer. The sequence of events in the chain is S1-R1-S2-R2+, where “+” signifies the primary reinforcer. Heterogeneous chains have been of interest because they begin to capture a feature of many natural behaviors, including addictive behaviors, which often presumably occur in chains. For example, a behavior such as smoking is not just the act of inhaling a cigarette. Instead, consumption occurs at the end of a set of other linked behaviors; a smoker who smokes in his or her car might first enter the car, turn on the radio, and then light a cigarette. In natural chains like smoking, each response in the chain is arguably under the control of its own discriminative stimulus (S), which is an important feature of a discriminated heterogeneous chain. Understanding the properties of discriminated heterogeneous chains may provide unique insights into treating instrumental behaviors (Ostlund & Balleine, 2008; Thrailkill & Bouton, 2016a).

Previous research has produced much evidence concerning the associative structure of the chain. For example, the results suggest that rats trained with an S1-R1-S2-R2 sequence come to associate the two behaviors. If R1 is extinguished (by separately presenting S1 and allowing the rat to perform R1 without receiving S2), R2 is weakened (Thrailkill & Bouton, 2015), and if R2 is extinguished (by separately presenting S2 and allowing R2 to occur without the pellet consequence), R1 is likewise weakened (Thrailkill & Bouton, 2016b). Recent research has also examined the contextual control of discriminated heterogeneous chains (Thrailkill, Trott, Zerr, & Bouton, 2016). Prior research had uncovered an important role for background context in controlling non-chained instrumental behaviors (Bouton, Todd, Vurbic, & Winterbauer, 2011; Thrailkill & Bouton, 2015; Todd, 2013). Consistent with the earlier work, when R1 and R2 were extinguished alone (outside of the chain), ABA and AAB renewal effects were observed (Thrailkill, Trott et al., 2016, Experiments 1 and 2). For example, rats were trained to perform a chain in a specific physical environment (Context A) and then given extinction training of R1 and R2 outside of the chain in a separate context (Context B). When placed back into Context A, R1 and R2 returned (ABA renewal). Interestingly, the context switch from A to B decreased the rate of R1, as it does simpler discriminated operants (Bouton, Todd, & León, 2014), but it had no apparent effect on R2. Thus, R1 was controlled at least in part by the background physical context, whereas R2 was evidently not. However, when the entire S1-R1-S2-R2 chain was tested in Context B after training in Context A, both R1 and R2 were decremented. The effect on R2 was presumably due to the fact that R1 had been weakened – as if R1, rather than the physical environment, was the “context” for performing R2.

To pursue this idea further, Thrailkill, Trott et al. (2016, Experiments 3 and 4) also examined contextual control of R2 by R1 by testing for a new renewal effect. After first training the chain, R2 was extinguished separate from the chain (by presenting S2 and allowing the rat to perform R2 without reinforcement). When R2 was then returned to and tested in the chain (i.e., S1-R1-S2-R2), the rat performed R2 again-- it was renewed. Importantly, the animal needed to be able to perform R1 for renewal of R2 to occur: Tests of S2-R2 after S1 only (with R1’s manipulandum removed from the chamber) did not produce renewal. Thus, R1 (rather than merely S1) functioned as the “context” for R2. To return to the example of the smoker who might light a cigarette (R2) after turning on the radio (R1), extinguishing smoking in the car might be effective, but the smoking response (R2) might renew if the person happens to turn on the radio (R1) again.

The present experiments were designed to extend the findings of Thrailkill, Trott et al. (2016). If R1 is the context for R2, then extinguishing R1 before the return-to-the-chain renewal test should weaken or eliminate the renewal of R2. Here, we tested that prediction in two experiments, the designs of which are summarized in Table 1. In both, we first trained rats to earn reinforcers by performing a discriminated heterogeneous chain in which R1 and R2 were pressing a lever and pulling a chain (counterbalanced). In Experiment 1, we asked if we could attenuate renewal of R2 following its return to the chain by also extinguishing R1 outside the chain. In Experiment 2, we asked whether performance of R1 during its extinction training is necessary to suppress the R2 renewal effect, and compared the effectiveness of extinction of R1 with simple unreinforced Pavlovian presentations of S1 without the opportunity to make R1 during the extinction phase.

Section snippets

Experiment 1

As noted above, Experiment 1 was designed to further test the role of making the procurement response (R1) in the renewal of consumption responding (R2) when the consumption response is returned to the chain by extinguishing R1 prior to the renewal test of R2. Three groups of rats were trained to perform a heterogeneous instrumental chain (S1-R1-S2-R2+). On each trial, an S1 (click or tone, counterbalanced) signaled the opportunity to perform a procurement response (R1) and an S2 signaled the

Subjects

The subjects were 24 female Wistar rats (Charles River, St. Constance, Quebec). They were approximately 90 days old at the start of the experiment, and were individually housed and food-deprived at 80% of their original weights throughout the experiment. Experimental sessions were conducted during the light portion of a 16:8 light-dark cycle and at approximately the same time each day. Rats had unlimited access to water when in their home cages and were given additional food after experimental

Acquisition

The acquisition of R1 and R2 responding is presented in Fig. 1. There was a clear increase in R1 and R2 elevation scores over sessions. R1 and R2 elevation scores were analyzed using separate Group by Session ANOVAs. These revealed a significant effect of session for both R1, F(7, 147) = 15.01, MSE = 456.67, p < .001, and R2, F(7, 147) = 7.52, MSE = 961.04, p < .001, with no group effects or group by session interactions, Fs < 1.

Each response was under strong stimulus control by the final

Discussion

When R2 was extinguished separately from the chain, it was renewed when it was returned to the chain (Group No S1R1-), thus replicating the results of Thrailkill, Trott et al. (2016). The new result is that additional separate extinction of R1 abolished the renewal effect in Group S1R1-. This result is consistent with the idea that R1 serves as a “context” for R2. It is worth noting that the results of the extinction phase suggest that the extinction of R1 did not suppress R2 directly;

Experiment 2

The purpose of Experiment 2 was to replicate and explore the boundaries of the effect of R1’s extinction on the renewal of R2. Previous results with simple discriminated operant procedures suggest that extinction of an operant response requires that the animal make the response during extinction trials; Pavlovian extinction exposure to a discriminative stimulus, without the opportunity to make the response, can have little effect on the strength of the operant response (Bouton, Trask, &

Subjects and apparatus

The subjects were 24 female Wistar rats (for each experiment) from the same supplier as before. Their age, maintenance, and housing, as well as the experimental apparatus, were the same as in Experiment 1.

Procedure

Magazine and chain training followed the procedure used in Experiment 1. Rats were then randomly assigned to one of three groups, all of which received daily S2R2- extinction sessions following the procedure used in Experiment 1. All groups also received an additional session each day during

Acquisition

One rat from Group S1 Yoked in Experiment 2a was removed from the analyses due to an abnormally high level of R2 responding during the S2 extinction reminder portion of the test (z = 2.36). This left us with n = 7 in Group S1 Yoked for Experiment 2a.

Acquisition of R1 and R2 during training for both Experiment 2a and Experiment 2b is summarized in Fig. 4. Separate Group by Experiment by Session ANOVAs on R1 and R2 responding revealed that R1 elevations increased across session, F(7,

Discussion

As in the previous experiment, extinction of R1 reduced the renewal of R2 after extinction when R2 was returned to the chain. That result, seen in all three of the present experiments, continues to suggest that R1 serves as a crucial element in causing the renewal of R2. However, R2 renewal was also reduced in groups that received only Pavlovian extinction of S1. Our expectation was that extinction of S1 would not weaken R1, and thus would not effect the renewal of R2. However, the results

General discussion

The present experiments were designed to test the possibility that renewal of R2 in the chain following extinction outside of the chain would be suppressed by extinguishing the R1 “context” of R2. Experiment 1 replicated previous findings that extinction of R2 outside the chain renewed when it was placed back into the chain (Thrailkill, Trott et al., 2016), but this renewal was prevented by extinguishing R1 in addition to R2. Importantly, extinction of R1 did not directly suppress R2 during its

Author note

This research was supported by Grant R01 DA 033123 from the National Institute on Drug Abuse to MEB. JAA was visiting Vermont from the University of Jaén; his participation was supported by Grant FPU-13/03761 reference EST16/00337 from the Spanish Ministry of Science, Culture and Sport. EAT was supported by Grant K01 DA 044456 from the National Institute on Drug Abuse. Send correspondence to Mark E. Bouton, Department of Psychological Science, University of Vermont, Burlington, VT 05405-0134.

References (18)

There are more references available in the full text version of this article.

Cited by (3)

  • Inactivation of the prelimbic cortex attenuates operant responding in both physical and behavioral contexts

    2020, Neurobiology of Learning and Memory
    Citation Excerpt :

    In fact, R2 appeared to be sensitive to a change in the behavioral context (i.e., the response rate decreased when R2 was tested separately from R1). Further, after extinction of R2, R2 renewed when tested back in the behavioral context in which it had been trained (R1-R2) but not when it was tested in a non-trained behavioral context (R3-R2) (Steinfeld, Alcalá, Thrailkill, & Bouton, 2019; Thrailkill et al., 2016). Here, we test the hypothesis that the PL is a key brain structure for the processing of not only physical context but also behavioral context as defined above.

View full text