An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents

Humans and other animals evaluate the cost (including delay, effort, and risk) to get a reward, and then make their decision to choose a certain course of action. Decision-making deficits appear in numerous neuropsychiatric diseases, including schizophrenia (SZ), attention deficit hyperactivity disorder (ADHD), obsessive-compulsive disorder (OCD), Parkinson's disease (PD) and addiction¹. Studies on humans and monkeys revealed that several key brain regions are involved in decision making^2,3,4. Although primates engage in more complicated decision makings, rodents have been reported to be able to make adaptive decisions to survive in an environment where reward contingencies change frequently. In addition, the neural circuit mechanisms and molecular mechanisms underlying decision making can be thoroughly investigated in mouse models due to the availability of chemogenetic tools, optogenetic tools and genetically engineered mice. There are multiple tasks used in evaluating rodents' decision-making behaviors, including the attentional set-shift task, the effortful or delay-based T-maze task, the Iowa gambling task, the visual discrimination reversal learning task⁵, etc. Analogous T-maze cost-benefit protocols were originally developed by the Pierre group⁶ and have been used to examine the effects of two types of decision cost (delay and effort) on free moving rodents^7,8,9,10. The special advantage of this task is that animals do not have to be trained to press levers or dig in a bowl. Instead, animals make a choice between a high reward high cost option in one arm (the HRA) or a low reward low cost option in the other arm (the LRA). Therefore, this task is much easier to perform.

In the delay-based paradigm, a junction door is introduced once the experimental animal enters one of the goal arms, so that the animal is kept in the goal arm. If the animal chooses the LRA, the goal door on the LRA is retracted immediately and a small quantity of food is delivered. If the animal chooses the HRA, the goal door on the HRA is retracted after the required delay and a large quantity of food pellets is delivered (Figure 1A). In the effort-based paradigm, the HRA is obstructed by a barrier and animals must climb over it to obtain a large quantity of pellets (Figure 1B). Generally speaking, the delay-based paradigm is very useful to test the impulsivity of animal models and the effort-based one can help to figure out apathetic animals^2,4,11,12,13. Hitherto researchers have been performing this assay by manually counting the time delay, inserting and withdrawing doors, maneuvering the effort barrier, counting the pellet number, placing pellets into position, placing and returning animals, and recording animals' choices for every trial. These labor and time costs pose a severe experimental bottleneck for researchers, hampering widespread use of this behavioral assay. In current work, we developed a T-maze based setup to assess delay- or effort-based decision making of rodents, with full automation, standardization, and high-throughput capacity.

Apparatus

In collaboration with a commercial manufacturer (see Table of Materials), we developed a modified automated T-maze apparatus which used software-based instrument control (Figure 2). In particular, we introduced a "back door" and "back way" as compared to the traditional T-maze (Figure 2), so that animals could go back to the start point themselves and start a new trial. The T-maze is matte grey colored, and when the experiment condition and software are set properly, both black and white mice can be detected. It is comprised of three arms: one start arm and two goal arms, each 410 mm in length with V-shaped walls of 155 mm in height, a base of 30 mm in width and an open top of 155 mm in width. The V-shaped corridor can effectively prevent mice from jumping. In addition, the V-shaped corridor makes it easier to apply in vivo recording with cables.A start box is attached to the end of the start arm. A goal box is attached to the end of each goal arm. An automatic food dispenser is installed in each goal box to deliver a predefined number of sweet food pellets. The pellet intake is detected by an infrared sensor and is automatically recorded by a computer. Each goal box is connected to the start box by a straight corridor. Animals can autonomously return to the start box via the corridor once they finish a trial. There are sliding doors of 155 mm in height at the entrance and exit of start and goal boxes. Additionally, a sliding door is located at the entrance of each goal arm to prevent animals from moving backwards after making a choice (Figure 2A). All sliding doors are controlled by a computer and can be automatically opened and closed. A high sensitivity 1/2" charge coupled device (CCD) monocle camera is set above the apparatus to track the animals' behavior. The focal length of the lens is 2.8–12 mm. The position of camera is around 1.9 m high. Since the height of maze is 0.5 m from floor, the distance between camera and the maze is around 1.4 m (Figure 2B). The tracking data obtained from the CCD camera is used to live-control the T-maze, opening and closing the specific doors when animals enter certain regions of interest (ROIs). The barriers used for the effort-based paradigm are in the shape of a three-dimensional right-angled triangle (Figure 2C), which perfectly fits in the V-shaped walls, and are about 155 mm in height. Animals must scale the vertical side but are able to descend a 45° slope. The apparatus is illuminated at 100 lux during the experiment. Sugar pellets used in the experiment (see Table of Materials), and silica-gel (see Table of Materials) is used to keep the pellets dry.

All experimental protocols were approved by the Animal Care and Use Committees of the RIKEN Brain Science Institute.

1. Animal Preparation

1. Choose the sex, age, genotype and pharmacological treatments of experimental mice (or rats) depending on the experimental purpose.
   NOTE: Here we demonstrated the performance of four male C57B/6 mice of 2 months old.
2. House the mice in a room maintained under standard conditions (12 h light/12 h dark cycle, lights on between 8:00 A.M. and 8:00 P.M., 22 ± 1°C).
   NOTE: If the purpose is to compare the difference between two genotypes, group 4 mice per cage and include 2 mice of each genotype.
3. Handle the mice for 2 min/day for 5 days to familiarize them with human contact. Feed them with measured rations so that their body weight is roughly maintained about 80–85% of the free feeding weight throughout the experiment. Provide water ad libitum.
4. Habituate mice to the experimental room by transferring all mice from the mouse housing room to the experimental room 30 min before the experiment each day.
5. Start experiments at the same time each day to avoid the effects of circadian rhythms on animal performance.

2. Animal Habituation to the Maze

1. Start habituation to the maze simultaneously with mouse handling (2 min/day). Keep all doors open at this stage. Perform habituation for a total of 5 days.
2. On day 1, scatter the food pellets throughout the maze.
3. On days 2 and 3, scatter the pellets along the two goal arms.
4. On days 4 and 5, put the pellets only at the two goal boxes.
5. Everyday, after placing the pellets, place the mice in the start box of the T-maze in groups of four and allow the mice to explore the maze for 10 min.
   NOTE: Habituating the mice in groups of four will help them to learn from each other and speed up training.

3. Animal Discrimination of HRA from LRA

NOTE: This protocol includes both delay-based and effort-based decision-making tests. However, depending on the purpose, researchers can test only one of them, or both. Control software (Table of Material) is used to automatically control the T-maze setup for the following steps. If effort-based decision making will be tested, introduce barriers to both HRA and LRA in the forced arm entry phase. Then animals will be trained for both discrimination and barrier climbing simultaneously. The starving mice actively climb the barriers and after this phase, all of them can climb skillfully. Therefore, it is not necessary to start from a lower barrier with this protocol.

1. Forced arm entry phase
   1. Open the parameter registration window of the control software and set up parameters as follows (Figure 3).
      1. Choose the "phase option". Set the "trial number" to 10, so that each animal will go through 10 trials per day for 5 continuous days.
         Note: One can choose a different trial number in his/her own experiments.
      2. Set the "Duration" to 900 s so that the training of one mouse per day will not exceed 900 s. Set the "Default start delay time" to 3 s, so that the start door will open 3 s after the animal is detected in the start area. 
      3. Set the "pellet number" for the HRA and the LRA so that 4 pellets are always automatically dispensed in the HRA and 1 pellet is dispensed in the LRA.
         Note: In our experiments, we found that 1:4 is the best ratio when 10 mg sugar pellets are used. If we use 6 to 10 pellets, the mice cannot finish eating all of them and there will be omission happening.
      4. Set the "Delay time" to 0 s, so that there will be no delay for both HRA and LRA during this phase.
   2. Open the "ID registration" window of control software. Register ID of each individual mouse to software according to the location of the HRA, either left side or right side. (Figure 4).
      Note: The location should be counterbalanced with respect to genotype groups. For 50% of each genotype group, the HRA is always on the left and the LRA is always on the right. For the other 50%, the HRA is always on the right and the LRA is always on the left.
   3. Open the application window of the software, select "Decision Making" from the dropdown "Task" list. Input subject ID and select "Phase 2" from the "Phase" dropdown list. Select day number from the "Day" dropdown list. Press "OK" button to enter the experiment interface window. 
   4. In the experiment interface window, press "GetBG" to register the background information of the maze so that the animal will be accurately tracked regardless of the background of the environment. Press the "SESSION START" button (Figure 5). 
   5. Place the mouse in the start box and initiate the training by pressing the "start" button of the remote control.
      1. Note that the start door, one junction door (either left or right side) will automatically open after 3 s; once the mouse enters the junction area, the start door will automatically close.
      2. Observe that once the mouse enters the delay area (either left or right side), the junction door will automatically close and the goal door will automatically open.
      3. Observe that once the mouse takes the pellet, the back door and pre-start door will automatically open. Once the mouse enters the back area, the back door will automatically close.
      4. Observe that once the mouse enters the start box, the pre-start door will automatically close, and a new trial will start.
         Note: Within 10 trials of each day during this training phase, the software will automatically ensure that each mouse visits the HRA for 5 trials and the LRA for 5 trials.
   6. Clean the maze thoroughly every day.
2. Free arm entry phase
   1. Register parameters and Subject ID in the same way as done in the forced entry phase (step 3.1.1 and 3.1.2). Choose the "phase option". Set the "trial number" to 20, so that each animal will go through 20 trials, per day for 7 continuous days.
   2. In the application window, select "Phase 3" from the "Phase" dropdown list. Set other parameters as per step 3.1.3.
   3. In the experiment interface window, set the value of "Success Rate" as 80% so that the training will automatically continue until the mouse selects the HRA in 80% of trials, or when the mouse finishes 20 trials per day (as is registered in parameters setup). Apply other operations as per step 3.1.4.
   4. Allow the mouse to freely choose one arm, either HRA or LRA. 
      1. Note that the start door, two junction doors will automatically open after 3 s; once the mouse enters the junction area, the start door will automatically close.
      2. Observe that once the mouse chooses one arm and enters the delay area (either left or right side), the junction door will automatically close and the goal door will automatically open.
      3. Observe that once the mouse takes the pellet, the back door and pre-start door will automatically open. Once the mouse enters the back area, the back door will automatically close.
      4. Observe that once the mouse enters the start box, the pre-start door will automatically close, and a new trial will start. 

4. Delay-based Decision-making Test

1. Register parameters and Subject ID in the same way as done in the free arm entry phase (step 3.2.1). Set the "Delay time" to 5, 10,15 s on day 1, day 2 and day 3 respectively, so that there will 5 s delay for HRA on day 1, 10 s delay for HRA on day 2, and 15 s delay for HRA on day 3. 
2. In the application window, select "Phase 4" from the "Phase" dropdown list. Set other parameters in the same way as in 3.2.2.
3. In the experiment interface window, apply all the operations as per step 3.2.3.
4. Allow the mouse to freely choose one arm, either HRA or LRA. 
   1. Note that the start door, two junction doors will automatically open after 3 s; once the mouse enters the junction area, the start door will automatically close.
   2. Observe that once the mouse chooses one arm and enters the delay area (either left or right side), the junction door will automatically close.
      Note: If the mouse chooses LRA, the goal door will automatically open immediately. However, if the mouse chooses HRA, the goal door will automatically open after 5 s, 10 s, and 15 s on days 1, 2, 3 respectively. 
   3. Observe that once the mouse takes the pellet, the back door and pre-start door will automatically open. Once the mouse enters the back area, the back door will automatically close.
   4. Observe that once the mouse enters the start box, the pre-start door will automatically close, and a new trial will start. 
      Note: Here, we trained the mice for 5–7 days with each delay condition. However, based on our experience on testing multiple lines of transgenic or mutated mice, 1 day (20 trials) is absolutely enough to see the difference between the mice of different genotypes and there is no meaning to extend the training time (See Figure 6 as an example). Therefore, currently we only apply 1 day for each delay time and it works well. There will be no problem if researchers want to elongate the training days depending on their own purpose.
5. Optional: Perform the test with the HRA reversed. To test if the mouse’s choice is the result of an orientation preference, switch the left/ right position of the HRA and the LRA (which can be accomplished automatically by the software) and allow the mice to freely choose one arm as in 4.4. 
6. Optional: Perform a delay control test. To test whether any deficit observed is the result of altered spatial memory or reward sensitivity rather than the result of changes in decision making, introduce a 15 s delay to the LRA as well as the HRA, and allow the mice to freely choose one arm as in 4.4. 

5. Effort-based Decision-making Test

1. Introduce the barrier to the HRA as shown in the diagram (Figure 1).
2. Set up all the parameters and apply all the operations as per step 3.2 — free arm entry phase and test the animals for 3 continuous days.
3. Allow the mice to freely choose one arm, either HRA or LRA.
   NOTE: Here, we trained the mice for 14 days. However, based on our experience on testing multiple lines of transgenic or mutated mice, 3 days are absolutely enough to see the difference between the mice of different genotypes and there is no meaning to extend the training time (See Figure 6 as an example). Therefore currently we only apply 3 days for effort-based test and it works well. There will be no problem if researchers want to elongate the training days depending on their own purpose.
4. Optional: Perform the test with the HRA reversed. To test if the mouse’s choice is the result of an orientation preference, switch the left/ right position of the HRA and the LRA (which can be accomplished automatically by the software) and allow the mice to freely choose one arm as in step 5.3.
5. Optional: Perform effort control test. To test whether any deficit observed is the result of altered spatial memory or reward sensitivity rather than the result of changes in decision making, introduce a barrier to the LRA as well as the HRA, and allow the mice to freely choose one arm as in step 5.3. 

6. Data Analysis

1. Obtain data and results directly from the control software.
   1. Note that the software will automatically record experimental date, start and finish time, duration, trial number, the location of the HRA, pellet number in the HRA and the LRA, the position (X, Y), and the moving trace etc., of each mouse in the "Data" folder.
   2. Check that the software has automatically analyze the following items and record them in the "Result" folder under each animal ID: Duration, trial number, HRA choice number, LRA choice number, HRA choice percentage, LRA choice percentage, total moving distance, and total junction time.
2. Perform statistical analysis on the data from all experiments by a mixed ANOVA (split-plot ANOVA), with day/session as within-subject factor and group factor (genotype group or groups with different experimental conditions) as between-subject factor.
3. Analyze the main effect of group factor if there is no interaction between day/session with group factor. Apply post hoc pairwise comparisons if there is a significant interaction between day/session with group factor.

An example of the delay- and effort- based decision-making task performed by medial habenular ablated mice (mHb:DTA mice)¹⁴ with their wild-type littermate control mice (CT mice) is shown in Figure 6. Two mHb:DTA mice and two CT mice were co-housed in one cage after weaning.

In the delay-based decision-making test (Figure 6A), there was no significant interaction between genotype and session in any phase, including the discrimination training phase (when the delay time of HRA was 0) and delay-based decision-making test phase (when the delay time of HRA was 5 s, 10 s, and 15 s, respectively). The main effect of genotype was not significant when the delay time was 5 s. However, when the delay time was elongated to 10 s and 15 s, mHb:DTA mice demonstrated a significant reduction in the percentage of HRA visits compared to CT mice. These results revealed that the ablation of mHb decreased the preference of mice to wait for a bigger reward, and instead displayed a tendency to select a small reward immediately, when the waiting times were 10 seconds or even longer. The data suggested that mHb might be an important brain structure in the control of impulsivity and/or time cost/benefit evaluation, rendering animals more prone to tolerate delayed access to get a large reward.

In the effort-based decision-making test (Figure 6B), the percentage of HRA visits were significantly decreased in mHb:DTA mice when a barrier was placed in the HRA, regardless of the left/right localization of the HRA (1x barrier phase and reversal phase). This means that the phenotype of mHb:DTA mice was not due to a deficit in spatial preference and memory. In the effort control test, barriers were placed in both goal arms (2× barriers phase) and both LRA and HRA were associated with high effort. Therefore, the effort cost was the same for animals selecting either the low reward or the high reward. The mHb:DTA mice visited the HRA more frequently than the LRA, and reached a comparable HRA visit number on the final session (session 5). This result suggests that reward sensitivity and spatial memory in mHb:DTA mice was intact. The data elucidated that mHb may play an important role in effort cost/benefit evaluation, allowing animals to put in more work to acquire greater rewards.

Figure 1: Schematic diagram of traditional T-maze apparatus for decision-making test. (A) Delay-based decision-making test apparatus. Animals were placed in the start arm and chose between two goal arms, HRA and LRA. When animals chose the HRA, they had to wait (depending on the delay time in seconds) to get a larger reward. Researchers need to manually handle animals, pellets, and doors for each trial. (B) Effort-based decision-making test apparatus. Animals were placed in the start arm and chose between two goal arms, HRA and LRA. When animals chose the HRA, they had to climb over a triangular barrier to get a larger reward. Researchers need to manually handle animals, pellets, doors and barriers for each trial. Please click here to view a larger version of this figure.

Figure 2: Automated T-maze setup for decision-making test. (A) Top view of the automated setup. (B) Side View of the automated setup. (C) The 3D right-angle triangle barrier used for effort-based decision-making test, from left to right, are the side view, the opposite side view and the hypotenuse side view, respectively. Original technical photos edited with permission from the commercial manufacturer. GBL: goal box (left), GBR: goal box (right), GDL: goal door (left), GDR: goal door (right), DAL: delay area (left), DAR: delay area (right), JDL: junction door (left), JDR: junction door (right), BDL: back door (left), BDR: back door (right), CCD: charge coupled device camera). Please click here to view a larger version of this figure.

Figure 3: The Parameter registration window. Please click here to view a larger version of this figure.

Figure 4: The Subject ID registration window. Please click here to view a larger version of this figure.

Figure 5: The experiment interface window. Please click here to view a larger version of this figure.

Figure 6: Delay-based and effort-based decision making in mHb:DTA mice. (A) Delay-based decision-making test in mHb:DTA mice (mice were 12–14 months-old, n = 8/genotype). The percentage of HRA choice was comparable between genotypes when the delay time was 0 and 5 s, but significantly decreased in mHb:DTA mice when the delay time was 10 s and 15 s. When the delay time was 5 s, genotype×day interaction: F(1,14) = 0.594, p = 0.236; the effect of genotype: F(1,14) = 0.61, p = 0.45; when the delay time was 10 s: genotype×day interaction: F(1,14) = 37.5, p = 0.346; the effect of genotype: F(1,14) = 32.4, p <0.0001; when the delay time was 15 s: F(1,14) = 38.7, p = 0.243; the effect of genotype: F(1,14) = 31.6, and p ≤0.0001. (B) Effort-based decision-making test in mHb:DTA mice (mice were 12-14 months-old, n = 9/genotype). During 1x barrier phase, there was a significant interaction between genotype and session (genotype×day interaction: F(1,16) = 2.11, p = 0.015), and the post hoc pairwise comparison revealed that HRA% of mHb:DTA mice significantly dropped in all sessions. During the reversal phase, there was no significant interaction between genotype and session (genotype×day interaction: F(1,16) = 1.61, p = 0.08). mHb:DTA mice visited HRA significantly less than CT mice (main genotype effect: F(1,16) = 8.18, p = 0.01). During 2×barriers phase, there was a significant interaction between genotypes and sessions, and a significant difference in session 3 and session 4 (2x barriers phase: genotype×day interaction: F(1,16) = 3.9, p = 0.0067). The mHb:DTA mice reached a HRA visit number comparable to that of CT mice on the final session, session 5. Data represent mean ± SEM. ** p <0.01; *** p <0.001. Please click here to view a larger version of this figure.

Figure 7: The flowchart of decision making test (delay- or effort-based).

Decision making is a cognitive process highly conserved during evolution¹⁵. Humans and animals can evaluate the cost of competing action options relative to the potential reward and then make their choice. Patients suffering from a number of neurological diseases and psychological disorders demonstrate deficits in different forms of decision making¹⁶. It is therefore important to investigate the neurobiological and pathophysiological mechanisms underlying the decision-making process. In the past few years, delay- and effort-based decision making is attracting more and more research interest. Furthermore, rodents, especially rats have been extensively used to study these two forms of decision-making¹⁷.

Many studies led to interesting discoveries using a behavioral task involving a T-maze apparatus with a HRA and a LRA^{2,6,7,8,9,10,18,19,20,21,22}. In the task, HRA associates large rewards with either a time delay or effort exertion. On the LRA, animals can acquire a small reward immediately without any time delay and physical effort. The traditional approach relies on human experimenter's manual intervention. In each trial, the experimenter needs to count the pellets and place them in the food trays of HRA and LRA, place the goal doors on both HRA and LRA, and then place the animal at the end of the start arm. When the animal enters either of the arms, a junction door needs to be placed to restrict the animal to the goal arm. Depending on the protocol, the experimenter needs to count the time and open the goal door after a set delay. After the animal enters the goal area and obtains the pellet(s), the experimenter needs to return it to the cage, and record the animal's arm choice and behavior. Then the experimenter needs to prepare the T-maze doors and pellet for the next trial. The whole training and testing processes are tremendously time and labor intensive. Furthermore, a lack of standardization across different labs is another concern.

In this paper, we presented a protocol based upon a modified automated T-maze apparatus with a video-tracking system (Figure 7) to solve the problems of traditional protocols. By introducing a "back door" and "back corridor" to the traditional T-maze, we obtained maze with a "bisected isoceles triangle" shape. The advantages of this setup are (1) full automation of the behavioral training and testing. This removes the impact of experimenter subjectivity and minimizes human time and labor commitments. We have four setups in the lab, so that four mice could be trained or tested simultaneously by one experimenter, which is impossible to be accomplished using traditional protocols. (2) There is software flexibility as the control software allows experimenters to freely set up multiple parameters, including pellet number, delay time, door opening and closing, trial numbers, duration, and trace mode. Therefore, this system can meet different kinds of experimental needs. (3) There is broad compatibility as all sliding doors on the T-maze are designed to be stored under the base of the maze when they are open. Therefore, the setup can be easily integrated with diverse physiological systems, including optogenetic/optical manipulation, in vivo electrophysiology recording, and microdialysis. In addition, for excluding the possibility that the mice chose the HRA due to a position preference, we recommend applying a control test for both the delay- and effort- based assay. By equalizing the costs in the two goal arms, animals have the opportunity to experience both reward outcomes at the same cost. The choice can be made simply on the basis of the reward differential, thus removing the need to integrate both costs and benefits before deciding. This also tests whether any change in the animals' choices is the result of an inability to scale the cost or reward, or memory deficit rather than an alteration in the way in which they assessed their decisions.

In our lab, we have analyzed about 10 strains of mice with this setup. One example was shown in the representative data, mHb:DTA mice demonstrated a robust phenotype in both delay- and effort- based decision making. That is, reward value is strongly discounted by time and effort in mHb:DTA mice. The result revealed the important role of mHb on impulsivity control. In addition, we have applied silicon probe recordings on free moving mice during the decision-making process (unpublished data). All the experiments provided validation benchmarks for the capability of the automated setup. Thus, the standardized protocol for the T-maze based decision making with the automated apparatus is suitable for detecting genetic effects, pharmacological effects and neural circuit effects on delay and effort discounting of rodents. In summary, the setup has many advantages to serve as an ideal system for the delay- and effort-based decision-making assays.