Pavlovian Conditioned Approach Training in Rats

The transition of drug use to addiction involves complex interactions between Pavlovian and instrumental learning^1,2. During drug-taking and drug-seeking behaviors, actions and outcomes are learned through instrumental processes; however, relationships between stimuli (e.g., drug-related cues) and rewards are also learned through Pavlovian processes. Pavlovian cues acquire predictive value, but they can also acquire incentive motivational value³, whereby they become attractive, desirable, and capable of promoting and maintaining reward-seeking behavior⁴.

Many procedures investigating reward learning in animals, however, do not permit the disentanglement of predictive versus incentive motivational learning. In drug self-administration procedures, for example, both instrumental and Pavlovian contingencies are typically employed, such that rats learn to perform an action (e.g., nose-poking, lever-pressing, etc.) in order to receive an outcome (i.e., intravenous drug infusion). The rewarding outcome is also paired with Pavlovian stimuli (e.g., illumination of a nose-poke port, cue light presentation, sound of the infusion pump, interoceptive feeling of fluid delivery into the bloodstream, etc.). It is often unclear in these procedures whether cues are supporting goal-directed actions simply because of their predictive relationship with the reward, or whether cues have acquired incentive motivational properties of their own.

In order to experimentally isolate the incentive motivational value from the predictive value of Pavlovian stimuli, our laboratory uses a Pavlovian conditioned approach (PCA) procedure in order to identify rats that attribute incentive salience to reward-related cues. During a training session, a conditioned stimulus (CS; e.g., a lever) response-independently predicts the delivery of an unconditioned stimulus (US; e.g., a food pellet). Over the course of multiple training sessions, three phenotypes develop: sign-tracking (CS-directed conditioned response [CR]), goal-tracking (US-directed CR), and an intermediate response (both CRs). For goal-trackers (GTs), the CS is utilized as a predictor of reward delivery; however, for sign-trackers (STs), the CS is attributed with incentive salience, becoming attractive and desirable. In this review, we outline the equipment, setup, and data processing necessary to perform a PCA procedure. In addition, we provide representative results of PCA training, outline important experimental considerations, and discuss the putative utility of PCA procedures in investigating addiction and other neuropsychiatric disorders.

All procedures have been approved by the University Committee on the Use and Care of Animals (UCUCA; University of Michigan, Ann Arbor, MI).

1. Equipment and Software

1. Obtain commercial operant chambers and devices. See Materials Table.
2. Conduct training in rat operant conditioning chambers (24.1 cm width × 20.5 cm depth x 29.2 cm height), situated in sound-attenuating cabinets with ventilation fans to provide ambient noise.
3. Equip each chamber with a pellet magazine (receptacle), located centrally in the 21.6 cm wide front wall of the operant chamber 3 cm above the stainless steel grid floor. Ensure that the pellet magazine contains an infrared sensor that is capable of detecting head entries into the device.
4. Connect the pellet magazine to a pellet dispenser, which delivers banana-flavored food pellets (45 mg).
5. In a counter-balanced manner, position one retractable lever (CS) approximately 2.5 cm on either side of the pellet receptacle (left or right) 3 cm above the stainless steel grid floor.
6. Ensure that the lever contains an LED cue light within it to make the extended lever more visually conspicuous.
7. If using a retractable lever, calibrate the lever by placing a 10 g calibration weight on top of the lever, and then adjusting the calibration screw (located on top of the device) with a hex nut wrench until the weight by itself is able to depress the lever.
8. Equip a red house-light at the top of the chamber on the wall opposite the magazine receptacle.
   Note: A red house light is used to increase the conspicuousness of the illuminated lever.

2. PCA Training

1. Pair-house adult male Sprague Dawley rats (250-300 g) upon arrival and leave them undisturbed for at least two days in the housing colony, which is maintained on a 12 hr light/dark cycle.
2. Gently handle each rat for at least 1 min per day for two days in order to acclimate the rats to human contact. Provide rats ad libitum access to food and water in the housing colony.
3. Familiarize rats with the banana-flavored food pellets by placing food pellets in their home cages (approximately 15 pellets per rat) for two days, starting three days prior to PCA training.
   Note: If rats are pair-housed, it is useful to divide the pellets evenly on two sides of the home cage.
4. Pretraining Program
   Note: All of our programing is performed using MEDState Notation (for Med Associates, software), and the manual is available online (mednr.com/pdf/doc003_MedPCProgramr.pdf). Programs are written as unformatted text documents (e.g., ASCII or DOS text) using MedState Notation, saved with the proper extension (i.e., [filename].mpc), then translated using the MED-PC program, Trans IV. All of these steps are provided in more detail in the aforementioned programming manual.
   1. Create a pretraining program that delivers fifty food pellets on a variable time (VT) 30 sec schedule 5 min after the start of the program (i.e., one food pellet is delivered on average every 30 sec, but the actual delivery varies randomly between 0-60 sec).
   2. Write the pretraining program such that the red house light is on, but the lever remains retracted for the duration of the (approximately 25 min) program.
   3. Perform pretraining and all subsequent experimentation during the light cycle.
   4. One day prior to PCA training, place each rat into an operant chamber, close and latch the chamber door, and close the sound-attenuating cabinet door.
   5. Start the pretraining program as soon as possible after placing each rat in its chamber.
   6. Remove each rat from its chamber as soon as possible after the program ends.
   7. Record the number of pellets consumed and exclude rats that do not consume all of the pellets from further testing.
5. Create a PCA training program that delivers 25 trials in approximately 40 min, beginning 1 min after the start of the program.
   1. Write the program so that during each trial the lever is extended into the chamber and illuminated for 8 sec.
   2. Program lever retraction to be followed immediately by the response-independent delivery of one food pellet into the pellet magazine.
   3. Separate each trial on a VT 90 sec schedule (i.e., lever extension and subsequent food pellet delivery occurs on average every 90 sec, but the actual delivery varies randomly between 30-150 sec).
   4. Place each rat into a conditioning chamber, close and latch the chamber door, and close the sound-attenuating cabinet door.
   5. Start the PCA training program as soon as possible after placing each rat in its chamber.
   6. Remove each rat from its chamber as soon as possible after the program ends.
   7. Exclude rats that do not consume all of the pellets from further testing.

3. Data Processing

1. Transfer output information to a spreadsheet (e.g., a spreadsheet document).
2. Create a profile that extracts identifying information (subject number, start date, start time, and box number) as well as output information (number and latency of lever presses and magazine entries).
   Note: In the experiments, non-CS magazine entries are also extracted by the profile. Define non-CS magazine entries as entries into the magazine that occur outside of the 8 sec CS period and during the VT 90 sec (i.e., 60-120 sec) period during which the lever is not extended.
3. Further process the data by capping the lever press and magazine entry latencies at 8 sec (i.e., the duration of CS presentation). This can be achieved in spreadsheet through the IF function: IF(X >8,8,X).
4. Calculate scores for response bias (i.e., total number of lever presses and magazine entries for a session; [lever presses - magazine entries] / [lever presses + magazine entries]), latency score (i.e., average latency to perform a lever press or magazine entry during a session; [magazine entry latency - lever press latency]/8), and probability difference (i.e., proportion of lever presses or magazine entries; [lever press probability - magazine entry probability])⁵.
   Note: With these formulas, all scores should range between 1.0 and -1.0.
5. Calculate a PCA index score for each rat by averaging its response bias, latency score, and probability difference within each individual session.
   Note: The PCA index score should range between 1.0 and -1.0.

We have found that 5-7 daily sessions of PCA training is sufficient to phenotype rats as STs, GTs, and intermediate responders (IRs), although rats can be trained further based on the needs of a particular laboratory or experiment. Phenotyping is based on PCA index scores, which are calculated by averaging the response bias, latency score, and probability difference of individual sessions as previously described in the protocol. Phenotypes are determined by averaging the PCA index scores from the last two PCA training sessions (i.e., Sessions 4 and 5 if there are a total of five PCA training sessions). The PCA index scores behavior from 1.0 (absolute sign-tracking) to -1.0 (absolute goal-tracking), with 0 representing no bias (absolute intermediate response). Because the tendency to sign- vs. goal-track varies over a continuum, cut-offs are required to identify phenotypes, and our laboratory generally uses PCA index score cut-offs of -0.5 and 0.5. In other words, STs have PCA index scores ≥0.5, IRs have PCA index scores between -0.5 and 0.5, and GTs have PCA index scores ≤-0.5. Based on the needs of the individual experiment, however, different PCA index score cut-offs can be used. Figure 1 shows a typical progression of the number, latency, and probability of lever presses and magazine entries across PCA training sessions in the three phenotypes. Figure 2 shows typical distributions of PCA index scores (averaged across Sessions 4 and 5) and average PCA index scores across PCA training sessions after rats have been phenotyped. As can be seen in both figures, PCA behavior stabilizes between Sessions 4-5.

In addition to these data, non-CS magazine entries (i.e., magazine entries performed during inter-trial intervals when the lever is retracted) may also be analyzed. These data may be particularly useful in determining whether a manipulation affects total conditioned responding (not just conditioned responding during the CS period) or alters the timing of conditioned responding. Additional analyses, such as pellet retrieval latency (i.e., the time to enter the magazine and retrieve the pellet following the CS period) or omissions (i.e., not performing any CR during the CS period) can also be calculated based on the needs of individual experiments.

Figure 1. Number, latency, and probability of lever presses and magazine entries across five daily sessions of Pavlovian conditioned approach (PCA) training in sign-trackers (STs), intermediate responders (IRs), and goal-trackers (GTs). In this experiment, male Sprague Dawley rats (n = 48; 275-300 g upon arrival) were ordered from Harlan Laboratories (Barrier 206) and Charles River (Barriers C72, R04, and R09). Data presented as mean ± S.E.M. Please click here to view a larger version of this figure.

Figure 2. (A) The distribution of Pavlovian conditioned approach (PCA) index scores averaged across the final two PCA training sessions. Scores are calculated for each session by averaging the bias in the number, latency, and probability of lever presses and magazine entries across all trials within an individual session. (B) PCA index scores (averaged over the final two PCA training sessions) are used to phenotype sign-trackers (STs), intermediates (IRs), and goal-trackers (GTs). The average PCA index scores of Sessions 4 and 5 are used to classify rats as STs (score ≥0.5), IRs (-0.5 <score <0.5), or GTs (score ≤-0.5). Data presented as mean ± S.E.M. Please click here to view a larger version of this figure.

PCA training can be used to determine individual variation in the tendency to attribute incentive salience to conditioned cues, which has been argued to be an important component of addiction vulnerability. For example, it has previously been demonstrated that STs attribute incentive salience to both food and drug cues⁶, and both food- and drug-related cues activate similar brain regions within an amygdalo-striatal-thalamic circuit in STs⁷. Moreover, STs are more impulsive⁸, vulnerable to cue-induced reinstatement of drug-seeking behavior^7,9, and seek drug-related cues despite adverse consequences¹⁰.

When performing a PCA procedure, the selection of rats represents a critical step in optimizing phenotypic diversity. For example, there are marked differences in the acquisition of PCA behavior in inbred strains of rats (e.g., Lewis and Fischer F344)¹¹. Even in outbred stocks (e.g., Sprague Dawley), there exists considerable heterogeneity in the acquisition of PCA behavior between vendors (e.g., Harlan Laboratories versus Charles River) and the individual facilities where rats are bred (i.e., Charles River R04 and R09 facilities in Raleigh, NC)¹². To increase phenotypic diversity, Sprague Dawley rats may be ordered from multiple vendors and barriers, as long as the exact source of each rat is carefully documented. The age and sex of rats should also be considered. For example, adults (compared to adolescents) and females (compared to males) tend to express more sign-tracking behavior^13,14.

It is important to note that housing conditions and conditioning parameters can limit the ability of this procedure to provide the desired level of phenotypic diversity during the acquisition of PCA behavior. In regards to housing conditions, food deprivation and social isolation alter the acquisition of PCA behavior, which also depends on the age of the rat¹⁵. If food deprivation is required (e.g., for subsequent operant conditioning), it should be performed following PCA training. In addition, environmental enrichment reduces sign-tracking behavior and should be taken into account when performing PCA procedures¹⁶.

Conditioning parameters also affect the acquisition of PCA behavior, and we have carefully tailored each parameter of our sessions to promote equal distributions of STs, IRs, and GTs. For instance, the duration of CS presentation and its spatial proximity to the US can bias behavior toward sign- or goal-tracking, such that shorter CS presentations¹⁷ and smaller CS-US distances¹⁸ promote sign-tracking behavior. In addition, the CS modality can affect the acquisition of sign-tracking behavior. In example, rats do not readily sign-track to nose-poke ports or auditory tones^19,20. Even when using a retractable lever as the CS, care should be taken not to overtrain rats. Previously, it has been observed that after sufficient training sessions, sign-tracking behavior, which is a dopamine-dependent^21,22, becomes dopamine-independent²³ potentially affecting the results of pharmacological manipulations. Moreover, reward uncertainty (i.e., decreasing the probability of reward delivery following CS presentation) increases sign-tracking behavior, which can interact with the spatial proximity of the CS and US (e.g., increasing reward uncertainty biases sign-tracking towards distal rather than proximal CSs)^24,25. Finally, the inter-trial interval (i.e., the time between CS presentations), and whether it is fixed or variable, can impact conditioned responding²⁶.

PCA offers some distinct advantages over other tests of reward-related behaviors. For example, PCA uniquely allows separate measurement of the incentive salience and the predictive properties of reward-associated cues. PCA behavior can also be used to identify individuals vulnerable to certain addiction-like behaviors while avoiding the experimental confound of exposure to psychoactive drugs. Disadvantages of the PCA procedure include the multitude of variables outlined above that can affect the PCA behavior of experimental subjects. Reproducibility of results is dependent on accounting for these many sources of variance, some of which, such as genetic variability and subtle differences in rearing conditions at commercial vendor facilities, can be all but impossible for experimenters to control. Furthermore, the neurobiological underpinnings of PCA behavior are not as well understood as those of other widely studied appetitive behaviors.

In addition to investigating addiction vulnerability, PCA has also been used to explore individual variation in vulnerability to other neuropsychiatric disorders, such as posttraumatic stress disorder. For example, STs are more vulnerable to cued fear expression²⁷ and show incubation of cued fear following extended fear conditioning²⁸. Therefore, PCA procedures can be used to model individual variation in cue-directed behavior that may have relevance for multiple neuropsychiatric disorders and even comorbid disorders (e.g., addiction and posttraumatic stress disorder). In addition, it appears that GTs are more vulnerable to context-induced reinstatement of cocaine-seeking²⁹ following cocaine self-administration and contextual fear expression following fear conditioning²⁷. PCA procedures might be useful for assessing these unique variations in associative learning (cued versus contextual learning), which can better inform investigations into the neural substrates and signaling pathways that may underlie separate vulnerabilities to neuropsychiatric disorders in unique subpopulations of individuals. Therefore, PCA procedures are potentially valuable assays of behavior to investigate the attribution of incentive salience, cued versus contextual processing, and individual variation in vulnerability to neuropsychiatric disorders.