Human performance on a two-alternative rapid-acquisition choice task

Abstract

Davison and Baum [Davison, M., Baum, W. M., 2000. Choice in a variable environment: every reinforcer counts. Journal of the Experimental Analysis of Behavior 74, 1–24.] developed a concurrent-schedule procedure where, within each session, different reinforcer ratios were arranged across components separated by brief black-outs. Behaviour adapted quickly to the reinforcer ratios and reinforcers also had local effects on responding. This procedure has been used with pigeons and rats. In the present experiment, we adapted the Davison and Baum procedure to study the effects of reinforcement on human choice behaviour. Eighteen participants were presented with four different reinforcer ratios within a single 50-minute session. Mean sensitivity to the reinforcer ratios increased within components, and preference was greater for the just-reinforced response alternative immediately following reinforcer delivery, similar to the results from non-human experiments. Although there were limitations to the current procedure, the local time scale analyses are a novel way of examining human operant behaviour.

Introduction

Choice in a standard two-alternative concurrent-schedules procedure is often described by the generalised matching law (GML—Baum, 1974):

$$\log \left(\frac{B_1}{B_2}\right)=a\log \left(\frac{R_1}{R_2}\right)+\log c$$

where B₁ and B₂ are the numbers of responses made on alternatives 1 and 2, respectively, and R₁ and R₂ are the numbers of reinforcers obtained for making B₁ and B₂ responses, respectively. The parameter “log c” is a measure of inherent bias that the subject may have for responding on one alternative over the other, irrespective of changes in the reinforcer distribution. The parameter “a” measures the sensitivity of the subject's behaviour to the reinforcer ratio; that is, the extent to which the relative distribution of reinforcers (log R₁/R₂) influences the subject's relative distribution of behaviour (log B₁/B₂). Estimates of a are usually between 0.8 and 0.9 for non-human subjects (Baum, 1979). However, estimates of a from human studies of choice are typically lower (with greater variability) than non-human subjects, possibly due to methodological or procedural differences (Kollins et al., 1997, Mazur, 1991), previous learning histories, and the role of rule-governed and other verbal behaviour on human responding (Horne and Lowe, 1993).

Most concurrent-schedules research has studied the effects of varied reinforcer contingencies (e.g., rate, magnitude, immediacy—see Davison and McCarthy, 1988 for a summary) on steady-state behaviour. Often, subjects sit many sessions for each pair of concurrent-schedules until behaviour meets some form of stability criteria. Data from the last few sessions of each concurrent-schedule pair are then analysed from a molar (or global) perspective. Recently, there has been an increase in the use of “rapid-acquisition” choice procedures, where unsignalled changes in reinforcer contingencies occur rapidly within sessions (e.g., Davison and Baum, 2000, Davison and Baum, 2002, Krageloh and Davison, 2003, Landon et al., 2003). Data from these procedures are analysed on a more local time scale, specifically looking at the local effects of reinforcement on choice behaviour.

In the first of a series of these experiments, Davison and Baum (2000) arranged a procedure where seven different pairs of concurrent VI schedules were presented in random order within each session (reinforcer ratios 27:1, 9:1, 3:1, 1:1, 1:3, 1:9, and 1:27). Each component (which comprised a single reinforcer ratio) was in effect for a set number of reinforcers, and following the completion of a component, there was a 10-s black-out period before the next component began. The session ended when the subject had completed all seven components or after 45 min, whichever occurred first. The pigeons experienced 50 sessions in total and data from the last 35 were analysed. Davison and Baum found that sensitivity to the reinforcer ratio (a) increased from near zero at the start of a component to moderate levels (approximately 0.6) after only around six reinforcers in a component, showing rapid acquisition of preference for the richer alternative in accordance to the generalised matching law (Eq. (1)). In a later study, Davison and Baum (2002) also investigated the local effects of reinforcement on preference. They found that immediately following a reinforcer, subjects showed a large preference towards the just-reinforced alternative (termed a “preference pulse”). Preference gradually decreased to indifference around 20 s following reinforcement. Preference pulses have also been found in standard concurrent VI schedule procedures (Landon et al., 2002). Finally, analyses of reinforcer sequences also showed that successive sequences of reinforcers for the same alternative (termed “continuations”) increased preference for that particular alternative (in a negatively accelerating manner), while a reinforcer obtained on the other alternative (a “discontinuation”) shifted preference towards the other (i.e., just-reinforced) alternative (e.g., Davison and Baum, 2000, Landon and Davison, 2001).

The Davison and Baum (2000) procedure has been used to study choice behaviour with pigeons (e.g., Davison and Baum, 2000, Davison and Baum, 2002, Landon et al., 2003) and rats (e.g., Aparicio and Baum, 2006, Breier et al., 2005). However, no published studies have used the procedure to study human choice behaviour. In fact, the Davison and Baum procedure seems ideal for studying the effects of varied reinforcer contingencies on preference with human participants because data can potentially be collected within a single session. The ability to collect adequate data over a short period of time is important for human research as attrition rates and decreasing levels of motivation can be problematic.

Like the non-human choice literature, most human choice experiments have looked at steady-state choice behaviour with data analysed following a number of training sessions. Although some experiments have employed procedures using within-session changes to the reinforcer contingencies (e.g., Bradshaw et al., 1976, Bradshaw et al., 1976, Bradshaw et al., 1977, Bradshaw et al., 1979, Horne and Lowe, 1993, Madden and Perone, 1999), these studies often use discriminative stimuli signalling the component in place (e.g., Bradshaw et al.’s experiments arranged discriminative stimuli that were ordinally related to the VI schedules used), components are separated by longer intervals (e.g., five minutes), and participants usually sit a large number of training sessions. Currently, no published studies have looked at the local effects of reinforcement on human choice behaviour or the effects of using a single session rapid-acquisition choice procedure.

The present experiment used a variation of the Davison and Baum (2000) procedure with human participants. Each participant was presented with four reinforcer ratios (5:1, 2:1, 1:2, 1:5) within a single session which lasted up to 50 min. We looked at the sensitivity of the participants’ behaviour to the reinforcer contingencies and the local effects of reinforcement. In particular, we were interested in whether humans showed similar patterns of responding as non-human subjects in the Davison and Baum procedure, and the suitability of the procedure for studying human choice behaviour.