Choice perseveration in value-based decision making: The impact of inter-trial interval and mood

Highlights

• People show choice perseveration in value-based decision making.

• Choice perseveration decreases with increasing inter-trial interval.

• Choice perseveration and its modulation are predicted by a neural attractor model.

Abstract

In a series of decisions, people tend to show choice perseveration, that is, they repeat their choices. This choice perseveration is assumed to emerge due to residual activity from the previous decision. Here, we use a computational model with attractor dynamics to describe this process and to predict how choice perseveration can be modulated. We derive two qualitative predictions: Choice perseveration should decrease under longer (vs. shorter) inter-trial intervals and positive (vs. negative) mood. We test these predictions in a dynamic decision task where we modulate decisions across trials via sequentially manipulated reward options. Our findings replicate our previous study in showing choice perseveration in value-based decision making. Furthermore, choice perseveration decreased with increasing inter-trial interval as predicted by the model. However, we did not find clear evidence supporting mood effects on choice perseveration. We discuss how integrating decision process dynamics by the means of applying the neural attractor model can increase our understanding of the evolution of decision outcomes and therefore complement the psychophysical perspective on decision making.

Introduction

Value-based and perceptual decision making in humans is often characterized by choice biases (Erev, Plonsky, Cohen, & Cohen, 2017; Hunt, 2014). Such choice biases can be split into two broad categories. One class subsumes biases that take effect due to properties of the actual decision such as framing, loss aversion, over-weighting of rare events (Erev et al., 2017; Kahneman & Tversky, 1979), and previous pay-off information (Noorbaloochi, Sharon, & McClelland, 2015). The second class subsumes biases that operate due to properties of the decision environment such as sunk costs (Bogdanov, Ruff, & Schwabe, 2015; Haller & Schwabe, 2014), a status quo (Nicolle, Fleming, Bach, Driver, & Dolan, 2011; Samuelson & Zeckhauser, 1988), and sequential effects caused by the responses in previous decisions (Bonaiuto, De Berker, & Bestmann, 2016; Hämmerer, Bonaiuto, Klein-Flügge, Bikson, & Bestmann, 2016; Scherbaum et al., 2016; Soetens, Boer, & Hueting, 1985; Soetens, Melis, & Notebaert, 2004).

In this paper, we put the focus on choice perseveration, that is, the tendency to stick to the present choice in future decisions. Choice perseveration and more complex sequential effects have been extensively studied in the context of perceptual decision making (Berlemont & Nadal, 2019; Cho et al., 2002; Fründ, Wichmann, & Macke, 2014; Gao, Wong-Lin, Holmes, Simen, & Cohen, 2009; Soetens et al., 1985). However, this important aspect is often neglected in empirical and modeling work studying value-based decision making, where trials of interest are analyzed in isolation. But why should sequential effects only occur in perceptual and not in value-based decisions? Both in real life and in the laboratory, people usually make many subsequent decisions. Thus, looking at the process dynamics of how one decision affects the next one can complement research on value-based decision making and inform future model developments. Here, we focus on choice perseveration as one example of first-order sequential effects (for an overview of other sequential effects and their mechanisms, see Gao et al., 2009).

We previously described such a dynamic process model that explains how choice perseveration emerges from sustained residual activity from the previous decision (Scherbaum et al., 2016). With the present study, not only aim to replicate this finding but we derive new predictions on choice perseveration and test them empirically. Before we turn to the new predictions, in the next paragraph, we will briefly describe the model as well as how it predicts and explains choice perseveration (for details, see Scherbaum et al., 2016, and the Appendix A).

The model is inspired by a value-based decision task in which people choose between rewards that are small in value and only a short distance away (small/near or SN option), and rewards that are large in value, but a larger distance away (large/far or LF option). For modeling purposes, we assume that people generate subjective values for each option. Taking this as a given, we will focus is on the process dynamics of how the decisions unfold over time (Spivey and Dale, 2004, Spivey and Dale, 2006; van Rooij, Favela, Malone, & Richardson, 2013).

In the model, options are represented as self-sustainable active patterns that constitute attractors in the neural system's state space (Reilly, 2006; Rolls, 2010; Scherbaum et al., 2016; Scherbaum, Dshemuchadse, Ruge, & Goschke, 2012). The dynamics of such models are often illustrated by conceiving of the system's state space in analogy to a potential landscape, where the stability of a state depends on the depth of the attractor (Rolls, 2010; Scherbaum, Dshemuchadse, & Kalis, 2008). In our model, the depth of the attractors is determined by the relative attractiveness of the SN and LF options, i.e. their relative subjective value. Fig. 1 depicts three kinds of possible attractor layouts. The left and right panel of Fig. 1 reflect almost exclusive activation of one option's representation (left panel: SN clearly more attractive; right panel: LF clearly more attractive), and hence illustrate configurations of the system with a preference towards one option. The middle panel of Fig. 1 represents a decision in which both options receive an identical input and are thus equally attractive; here, random noise would tip the system towards one or the other decision state (please see the Appendix A for the formal computational implementation of such a neural system).

A major advantage of such an attractor model is that it predicts dynamics of decision making on different time scales. In our model, the decision is not only determined by the current attractor layout, which is in turn determined by the currently offered options, but also by the history of the system's previous decisions (see Fig. 2) (Scherbaum et al., 2008; Townsend & Busemeyer, 1989). Due to the inertia of the model, the system temporally reclines in the area where it ended up previously—in the vicinity of the vanished attractor representing the recent choice—and relaxes only slowly to the neutral start state under no input. This process explains how choice perseveration emerges. For example, if the model chose the SN option in a first decision trial, it would remain near the SN option's attractor in the inter-trial interval. In a second decision trial, it would hence start the decision with a bias to the previously chosen SN option, even if this trial comprises the LF option being more attractive (see Fig. 2). This choice perseveration bias can be so strong that, in a series of choices where the initially unchosen option becomes increasingly more attractive, people switch to this now more attractive option much later than they would if their choices were unbiased by the previous decision. This phenomenon is called hysteresis or path-dependence (Rączaszek, Tuller, Shapiro, Case, & Kelso, 1999; Tuller, Case, Ding, & Kelso, 1994).

In our previous study, we demonstrated that the model indeed successfully predicts choice perseveration in human participants. Now, we identify potential modulators of choice perseveration and compare model predictions to human decision making. We focus on two modulators: the inter-trial interval (ITI) and mood, for which we derive the following hypotheses.

First, we expect to replicate our previous finding (Scherbaum et al., 2016) that people show choice perseveration in our value-based decision making task (H1).

Second, we expect the ITI to influence choice perseveration. The ITI determines how much residual activity from the previous decision is present at the start of the new decision. If the system had as much time as it needs to fully relax to the neutral start state between two options, no residual activity should be present at the time of the next decision, and choice perseveration should vanish. In contrast, if the system only had a short time span to relax before the start of the next trial, then the residual activity would be maximal, leading to strong perseveration effects. Thus, choice perseveration should decrease with increasing ITI (H2).

Third, we expect influences of mood.² A manipulation of mood should modulate choice perseveration because mood affects the overall flexibility of behavior. Specifically, positive mood has been linked to increased flexibility and decreased perseveration (Baumann & Kuhl, 2005; Dreisbach & Goschke, 2004; Nadler, Rabi, & Minda, 2010; Wang, Chen, & Yue, 2017; Zwosta, Hommel, Goschke, & Fischer, 2013). Goschke and Bolte (2014) suggest that this increased flexibility might be modulated by the dopaminergic system, for example through changes in the gain parameter (Servan-Schreiber, Printz, & Cohen, 1990), as both mood and gain have been associated with changes in the dopaminergic system (mood and dopamine: Ashby, Isen, & Turken, 1999; Dreisbach & Goschke, 2004; gain and dopamine: Herd et al., 2014; Li, Lindenberger, & Sikström, 2001). In the context of the attractor model, the gain parameter determines how strongly a system responds to input and how stable—and thus how hard to overcome—a certain state is (Herd et al., 2014; Servan-Schreiber et al., 1990). Thus, similarly to positive mood, a lower gain has been associated with decreased perseveration and increased flexibility (Eldar, Cohen, & Niv, 2013; Eldar, Niv, & Cohen, 2016; Herd et al., 2014). Thus, we expect positive mood to lead to decreased choice perseveration, possibly due to a decreased neural gain (H3).

Additionally, through the manipulation of people's mood, decision making in general might be modulated irrespective of perseveration (Phelps, Lempert, & Sokol-Hessner, 2014). Negative mood has been linked to more localized or focused, and hence more analytic processing (for a review see Mitchell & Phillips, 2007), which should increase participants' ability to detect optimal (i.e., best ratio of costs and benefits) choices in our value-based decision making task (H4a). Following from that, we further expect that participants in the negative mood group make more optimal choices than participants in the positive mood group (H4b).

We have simulated both the ITI and the gain manipulation in our neural attractor model (see Appendix A) to validate our verbally derived hypotheses for choice perseveration. As expected, the model simulations show that choice perseveration decreases with increasing ITI and with decreasing gain.