What’s flexible in behavioral flexibility?

Abstract

Behavioral ecologists interested in comparative cognition have struggled to design tasks that are both ecologically relevant and experimentally rigorous. In experimental psychology, standardized tests of reversal learning, set-shifting and self-control have long been used to measure aspects of flexible behavior especially with regards to determining the neural mechanisms that enable animals and humans to rapidly and efficiently adapt to different situations. More recently, behavioral ecologists have used the term “behavioral flexibility” more broadly to explain differences in traits such as personality and innovation. Here, we argue that the term behavioral flexibility designates too many non-equivalent traits, and that this can lead to misconceptions about the nature of cognitive abilities.

INTRODUCTION

The terms “behavioral flexibility” and “cognitive flexibility”, used interchangeably, have been employed for decades in the field of experimental psychology to label a form of cognition that enables animals and humans to adapt their behavior to changing environmental contingencies (e.g. Grattan and Eslinger 1989; Ragozzino et al. 1999; Floresco et al. 2009). In experimental psychology, commonly used tests of behavioral flexibility include reversal learning, set-shifting and self-control. In recent years, the term has featured prominently in behavioral ecology, where it is sometimes applied in the same manner as in psychology via tests of reversal learning, but more often in the context of innovation (Sol et al. 2002; Reader and Laland 2003) and problem-solving (Leal and Powell 2012). However, there is increasing evidence that innovative problem-solving and reversal learning are distinct, if not opposite, abilities (e.g. Griffin et al. 2013), while within psychology, different tests of flexibility may well be measuring different traits (Griffin and Guez 2014). If we add to this the many unrelated behaviors that the term has been applied to (e.g. animal personality: van Overveld and Matthysen 2013); defense mechanisms: Stoekl et al. 2015); division of labor: Kwapich and Tschinkel 2016), there is a clear risk that behavioral flexibility as a concept, let alone a term, will completely lose its significance.

Here, we briefly survey the ways in which behavioral flexibility has been assessed and conclude that even if the different assays used in behavioral ecology are conceptually linked, there is little empirical evidence that they are related. We argue that referring to such a large number of potentially non-equivalent and non-related skills with a single term is not useful, often misleading and should be avoided.

BEHAVIORAL FLEXIBILITY IN PSYCHOLOGY

In experimental psychology, the concept of behavioral flexibility emerges from principles of animal learning (for reviews, see Sutherland and Mackintosh 1971; Dickinson 1981) in which an animal makes a decision or choice that is largely influenced by various schedules of reward and future reward outcomes (see Clarke et al. 2004; Chudasama 2011). One commonly used scenario, and one that has been readily adopted in comparative studies, is the reversal learning paradigm, where a dominant response must be overridden due to changes in reward contingencies (see e.g. Jones and Mishkin 1972; Rolls 2000). First, the animal associates one rewarded conditional stimulus (CS+) with a response leading to a reward in the presence of a second, unrewarded stimulus (CS−). This process may continue over several hundred trials, encouraging the formation of a dominant response. Then, unknown to the animal, the stimulus-reward contingency is reversed and the animal must now change its response and use the previously unrewarded stimulus as a cue. There is some response persistence to the initially rewarded stimulus, as would be expected, before the animal works out the new rule. In some cases, however, the response persistence may be exaggerated. This is the case, for example, of rats or monkeys (marmosets and macaques) with orbitofrontal damage, indicating that this structure is involved in reversal performance (Dias et al. 1996; Chudasama and Robbins 2003; Schoenbaum et al. 2003; Izquierdo et al. 2004; Kim and Ragozzino 2005; Jang et al. 2015).

Related to reversal learning is the set-shifting paradigm, where the animal’s attention is solicited by different stimulus dimensions and the animal must alternate between strategies, rules, and attentional sets (Roberts et al. 1988). The cues can be olfactory, tactile, visual and spatial at the same time. The subject needs to first focus on one type of stimulus (for example, a rewarded and an unrewarded stimulus that differ in color) to get the reward as in a classic discrimination learning task, but then it must switch to another stimulus dimension (for example, spatial position or shape or texture) to distinguish the rewarded and unrewarded stimuli in the next phase, ignoring the previously rewarded color dimension (Dias et al. 1996; Oswald et al. 2001; McAlonan and Brown 2003; Brigman et al. 2005; Floresco et al. 2008). Therefore, in set-shifting, the rule is less tangible and the animal must form multi-dimensional attentional sets and shift between them to succeed. Although reversal learning and set shifting are related, they are anatomically dissociable: reversal learning, which involves adapting behavior in accordance with changes in stimulus-reward contingencies, requires an intact orbital prefrontal cortex in mammals, whereas switching attention between perceptual dimensions as in set-shifting relies on the lateral prefrontal cortex in primates or medial prefrontal cortex in rats (Chudasama and Robbins 2006; Nilsson et al. 2015). In sum, set-shifting tasks are designed to measure the subject’s ability to switch strategies, rather than simply learn a new association by reversing a previous one, and this is reflected in the different neural circuits that are involved in the two tasks.

Self-control is considered to be another aspect of behavioral flexibility both by experimental psychologists (see review by Coutlee and Huettel 2012) and behavioral ecologists (e.g. Amici et al. 2008; Boogert et al. 2011). Sometimes also referred to as “cognitive control”, self-control is defined as the extent to which an animal is able to withhold or inhibit its action in the face of a more immediate apparent reward. One way of testing for spatial self-control is the detour-reaching task, commonly used in comparative studies, which requires the animal to inhibit direct attempts to reach a visible food reward in a transparent apparatus, and to instead make a detour around the transparent obstacle to retrieve the food (Diamond 1990; Wallis et al. 2001). Self-control probably involves different brain areas than do reversal learning and set-shifting, at least in humans (see Aron et al. 2014). Although this is still a matter of debate, the right inferior frontal cortex seems to be one of the main areas responsible for self-control (Aron et al. 2014, but see Swick et al. 2008). In short, self-control tasks assess a subject’s ability to inhibit its initial response of using the simplest route or strategy to focus on an indirect, but more efficient approach, an ability that appears to be neurologically distinct from reversal and set-shifting tasks’ proficiencies.

Behavioral ecologists that look to psychology for standardized, well documented assays of animal cognition should thus be aware that, whatever the conceptual similarities between the tasks described above, there are thus clear differences in the traits that they measure (reversal of an association, attention to different cue dimensions, inhibition or impulse control), as well as their neural substrates (e.g. lateral prefrontal, orbitofrontal or right inferior frontal cortex; Wallis et al. 2001; Chudasama et al. 2003; Chudasama and Robbins 2003; Rudebeck et al. 2006; Kühn et al. 2009; Sharp et al. 2010; Aron et al. 2014). This heterogeneity needs to be taken into account when transposing tasks and their interpretation to the more naturalistic situations that behavioral ecologists usually focus on. For example, although detour-reaching and reversal learning are both said to measure flexibility, a study on wild-caught song sparrows found that the two tasks had “opposite” relationships with song repertoire: repertoire size had a positive relationship with detour reaching performance, but a negative one with reversal learning (Boogert et al. 2011).

BEHAVIORAL FLEXIBILITY IN ECOLOGY

Behavioral ecologists sometimes use the same tasks as experimental psychologists (e.g. reversal learning: Bond et al. 2007, detour reaching: Boogert et al. 2011), but they also often think of behavioral flexibility in terms of innovation and problem solving (e.g. Reader and Laland 2002; Sol et al. 2002; Tebbich et al. 2010; Wright et al. 2010; Huebner and Fichtel 2015). Innovation is defined in non-humans as a solution to a novel problem or a novel solution to an old problem (Kummer and Goodall 1985). Extractive foraging problems requiring obstacle removal have become a classic experimental test for innovation, following decades of studies on the origin and spread of the oldest (1921) animal innovation in the literature, the opening of milk bottles by tits (Fisher and Hinde 1949). While there is still a debate about the relative roles of persistence, motor diversity and cognition in the solving process (Griffin et al. 2014; Quinn et al. 2014; Rowe and Healy 2014; Thornton et al. 2014; Morand-Ferron et al. 2015; Cauchoix and Chaine 2016; Diquelou et al. 2016; Pritchard et al. 2016), there is some agreement that obstacle removal problems are a good way of assessing innovative foraging in experimental tests (reviewed in Griffin and Guez 2014). Studies on birds are the most numerous in this field (but see Thornton and Samson 2012; Benson-Amram et al. 2016). Overall, the studies suggest a negative (interindividual: Griffin et al. 2013, interpopulational: Tebbich and Teschke 2014, interspecific: Tebbich et al. 2010; Tebbich et al. 2012) or zero (interindividual: Boogert et al. 2011; Isden et al. 2013; Shaw et al. 2015; Logan 2016, interpopulational: Audet et al. 2016) relation between reversal learning and problem-solving performance. Likewise, problem-solving and detour reaching performance are often uncorrelated in birds (Boogert et al. 2011), but also interspecifically in great apes: orangutans are by far the best of the four great apes in detour-reaching (Vlamings et al. 2010), but the worst in an extractive problem requiring repeated innovation (Manrique et al. 2013).

By their very nature, reversal learning tasks might measure very different processes than the ones measured by innovation and extractive foraging problems. In a reversal task, there is a sudden change in the relationship between two cues and a reward, such that the cue that repeatedly predicted the reward in preceding trials is no longer predictive, and the cue that never predicted reward becomes highly predictive. In serial reversals, previously correct cues repeatedly become suddenly incorrect and previously incorrect cues repeatedly become correct. Persistence leads to errors in such tasks (see review by Nilsson et al. 2015), but persistence is on the contrary a strong facilitator of success in innovative problem-solving (Gajdon et al. 2006; Tebbich et al. 2010; Overington et al. 2011; Benson-Amram and Holekamp 2012; Cole and Quinn 2012; Thornton and Samson 2012; Cauchard et al. 2013; Huebner and Fichtel 2015; Griffin and Guez 2016). Chow et al. (2016) have recently tested the effects of persistence and flexibility on problem-solving efficiency in grey squirrels. They found that “flexibility”, measured as the rate of switching between tactics, was not linked to problem-solving performance, whereas persistence was a strong predictor of success. In addition, the sudden and often repeated changes in cue value in reversal and set-shifting tasks characterize neither extractive foraging problems in captivity or innovation cases in the wild. In fact, the problems that are solved in the wild are often very similar to captive extractive foraging problem-solving tasks but, to our knowledge, do not resemble reversal learning tasks.

While neural substrates of innovative problem-solving are still poorly understood, a few studies on laboratory rodents points to specific areas of the prefrontal cortex. In mice, inactivation of the medial prefrontal cortex causes deficits in an obstacle removal problem (Ben Abdallah et al. 2011) and in set-shifting, but not in reversal learning (Floresco et al. 2008), while in rats, the beta-adrenergic antagonist propranolol negatively impacts obstacle removal but not set-shifting (Hecht et al. 2014). This neurobiological evidence, together with correlational data, suggests that innovative problem-solving and other behavioral flexibility measurements are distinct proficiencies. As most recent studies of innovative problem-solving are done on birds, it should be noted that multiple lines of evidence point to the nidopallium caudolaterale (NCL) as the equivalent of the mammalian prefrontal cortex, providing a clear candidate structure for the control of similar behaviors in the 2 classes (Mogensen and Divac 1993; Rose and Colombo 2005; Rose et al. 2010; Herold et al. 2011; Helduser and Güntürkün 2012; Shanahan et al. 2013; Veit and Nieder 2013; Lengersdorf et al. 2015).

Innovating in the flexible usage of the term

While reversal learning, innovation and problem-solving dominate the literature on flexibility in behavioral ecology, the term has recently come to be used to qualify a surprisingly broad range of behaviors, including variation in neophilia/neophobia in primates (Bergman and Kitchen 2009), exploratory behavior in birds (van Overveld and Matthysen 2013), vigilance level in birds (Couchoux and Cresswell 2012), tool-use in primates (Vale et al. 2016), nest site choice in turtles (Barsante Santos et al. 2016), division of labor among colony members in ants (Bernadou et al. 2015; Kwapich and Tschinkel 2016) or between parents in frogs (Ringler et al. 2015), daily activity allocation in fish (Fingerle et al. 2016), niche allocation in rats, fish and birds (Igulu et al. 2013; Hunt 2016; Loveridge et al. 2016), courtship timing in spiders (Bardier et al. 2015), adjustment of feeder use in birds (Herborn et al. 2014), social organization in primates (Kamilar and Baden 2014; Otani et al. 2014), trial-and-error (discrimination, not reversal) learning in bats (Zhang et al. 2014), diversity of material used for nests in bees (MacIvor and Moore 2013), intensity of chemical defense in wasps (Stoekl et al. 2015), foraging activity across trials in fish (Adriaenssens and Johnsson 2011), degree of soft tissue retraction in foraging snails (Edgell et al. 2009), and the adjustable choice of suction or compression to process food items in elasmobranches (Wilga et al. 2012). This rich diversity of behavioral investigations is useful, as it provides a detailed picture of how behaviors are modified under changing conditions. However, there is little chance that all these cases share a similar etiology. Therefore, referring to this huge diversity of traits under the same blanket term is problematic. Within certain limits, a concept can be multifaceted, but the number and nature of the different contexts in which “behavioral flexibility” has been applied seems excessive. Based on the actual flexible usage of the term, flexibility is attributed to such a wide array of behaviors that are likely to have very different underpinnings that the term is more confusing than useful, especially in cognitive ecology. In this field, experiments implicitly or explicitly aim to understand the cognitive, and eventually neural processes, behind the behaviors tested. Given the mechanistic implications of such experimental studies, the use of a blanket term is especially problematic, as studies within both psychology and behavioral ecology point to heterogeneity in the co-variation and neural underpinnings of the different assays. In large scale comparative analyses of innovation in the wild (e.g. Reader and Laland 2002; Sol et al. 2002), where the focus is not on mechanisms, but on a wide variety of manifestations that go from simple incorporation of new foods in the diet to more sophisticated technical skills (Overington et al. 2009; Ducatez et al. 2015; Navarrete et al. 2016), the problem is less acute, but still preoccupying.

CONCLUSION

What’s flexible in behavioral flexibility? In brief, ways of measuring it. Our review suggests that different assays of behavioral flexibility used in experimental psychology and behavioral ecology are not necessarily equivalent, do not co-vary and are controlled by different neural mechanisms. Some of these assays are even designed to assess opposite abilities, given the contrasting effect of persistence on the performance of each task. Consequently, referring to innovative problem-solving, reversal learning, set-shifting, self-control or other even more distant traits under the same general term of behavioral flexibility can lead to misconceptions about how behavior should be interpreted, especially when comparing cognitive mechanisms across species. Thus, we suggest that the term should be avoided, at least in behavioral ecology. The precise tasks used to assess flexibility in experimental studies, whether they use the standard tasks and model species of psychology or the more naturalistic context of behavioral ecology, should be specified, as we gain more and more detailed knowledge of the mechanisms and the neural events that regulate the different ways in which animals change their behavior in the face of environmental challenges.

ACKNOWLEDGEMENTS

We thank Simon Ducatez and Yogita Chudasama for helpful discussions, as well as three anonymous reviewers for comments.

REFERENCES

Author notes