Elsevier

Neuropsychologia

Volume 46, Issue 5, 2008, Pages 1401-1414
Neuropsychologia

Effects of emotional and non-emotional cues on visual search in neglect patients: Evidence for distinct sources of attentional guidance

https://doi.org/10.1016/j.neuropsychologia.2007.12.027Get rights and content

Abstract

In normal observers, visual search is facilitated for targets with salient attributes. We compared how two different types of cue (expression and colour) may influence search for face targets, in healthy subjects (n = 27) and right brain-damaged patients with left spatial neglect (n = 13). The target faces were defined by their identity (singleton among a crowd of neutral faces) but could either be neutral (like other faces), or have a different emotional expression (fearful or happy), or a different colour (red-tinted). Healthy subjects were the fastest for detecting the colour-cued targets, but also showed a significant facilitation for emotionally cued targets, relative to neutral faces differing from other distracter faces by identity only. Healthy subjects were also faster overall for target faces located on the left, as compared to the right side of the display. In contrast, neglect patients were slower to detect targets on the left (contralesional) relative to the right (ipsilesional) side. However, they showed the same pattern of cueing effects as healthy subjects on both sides of space; while their best performance was also found for faces cued by colour, they showed a significant advantage for faces cued by expression, relative to the neutral condition. These results indicate that despite impaired attention towards the left hemispace, neglect patients may still show an intact influence of both low-level colour cues and emotional expression cues on attention, suggesting that neural mechanisms responsible for these effects are partly separate from fronto-parietal brain systems controlling spatial attention during search.

Introduction

While our gaze is freely scanning the surrounding environment, it may be caught by salient visual features that seemingly pop out of the background. Thus, on some occasions, our attention might be involuntarily drawn to a smiling face, or an odd-looking object, among a crowd of monotonous neutral stimuli. Likewise, in experimental situations, abundant research in normal subjects has shown that selective attention tends to be preferentially guided to salient or odd visual stimuli in a scene (Wolfe & Horowitz, 2004). Such effects have been observed not only for stimuli that differ from background items by a simple visual feature (e.g. colour or orientation), but also for stimuli with a particular emotional or social significance (Fox, Russo, Bowles, & Dutton, 2001; Fox, Russo, & Georgiou, 2005; Eastwood, Smilek, & Merikle, 2003; Lundqvist & Ohman, 2005; Williams, Moss, Bradshaw, & Mattingley, 2005). For instance, in visual search tasks, emotionally expressive face targets may capture attention faster (Hahn, Carlson, Singer, & Gronlund, 2006; Ohman, Flykt, & Esteves, 2001) and/or more effectively (Eastwood et al., 2003, Fox et al., 2001, Fox et al., 2005; Fenske & Eastwood, 2003), as compared with neutral faces. Similar behavioural effects of emotional cues on spatial attention have been found in exogenous orienting tasks derived from Posner paradigm (e.g., Fox et al., 2001; Pourtois, Grandjean, Sander, & Vuilleumier, 2004). However, the nature of these emotional influences on attention and visual search is still debated, as faces and expressions involve relatively high-level visual representations that may not be extracted as efficiently as the more simple feature responsible for classic “pop-out” effects (Pessoa, Kastner, & Ungerleider, 2002; Wolfe & Horowitz, 2004).

At the neural level, recent brain imaging work in healthy subjects has begun to uncover the possible neural correlates for such emotional facilitation of visual processing. Several studies have shown greater activation of extrastriate visual areas in response to emotional stimuli (e.g. fearful faces), as compared to neutral stimuli (Ishai, Pessoa, Bikle, & Ungerleider, 2004; Vuilleumier, 2007; Vuilleumier, Armony, Driver, & Dolan, 2001). Emotional stimuli also activate a network of specific limbic brain regions, such as the amygdala, orbitofrontal cortex, anterior cingulate, and insula (Haxby, Hoffman, & Gobbini, 2002). It has been suggested that modulatory feedback from limbic regions (in particular the amygdala) might act to increase activation of visual areas (Amaral, Behniea, & Kelly, 2003; Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004), and hence be responsible for preferential attention towards emotional visual stimuli (Vuilleumier, Armony et al., 2002; Vuilleumier, 2005). Accordingly, this emotional modulation might provide a separate top-down influence on visual processing that operates in parallel, perhaps additively, to the top-down influences exerted by classic attentional systems in fronto-parietal networks (Vuilleumier, 2005). Here we tested this hypothesis by investigating whether emotional cues might still influence visual search in patients with spatial neglect, who typically fail to direct attention towards the contralesional side of space due to unilateral damage to the fronto-parietal attention systems (Driver & Vuilleumier, 2001; Milner & McIntosh, 2005; see also Kastner & Ungerleider, 2001).

Spatial neglect is a frequent neurological syndrome, occurring most often after right hemisphere lesions, particularly in the parietal or frontal lobe. It is characterized by abnormal biases and restrictions in spatial attention, leading to severe impairments in the ability to perceive and/or explore stimuli on the side of space opposite to the brain lesion (most often left space). Neglect is typically observed during visual search tasks, such as clinical cancellation tests (Gauthier, Dehaut, & Joanette, 1989) where the patients must mark all targets present in a visual array, but fail to find those located on the contralesional half of the array, and explore the ipsilesional half repeatedly instead (Mannan et al., 2005). In these patients, the accuracy of stimuli in contralesional space is particularly slow, or even impossible, in the presence of competing distracters (Mark, Woods, Ball, Roth, & Mennemeier, 2004; Wojciulik, Rorden, Clarke, Husain, & Driver, 2004). This effect of competition is also illustrated by the phenomenon of perceptual extinction on double stimulation, where a visual stimulus in the contralesional hemifield is perceived when presented alone, but not when presented with another simultaneous stimulus in the ipsilesional field (Driver & Vuilleumier, 2001; Posner, Walker, Friedrich, & Rafal, 1987; Rafal, 1994). Thus, any competition for attentional resources will usually exacerbate the patients’ biases in attention or exploration, and further reduce awareness of contralesional stimuli.

Nevertheless, some categories of stimuli may compete more efficiently for attention and seem more likely to reach awareness when presented in contralesional space, despite the pathological neglect biases of these patients. This modulation of attention suggests that some residual processing may still take place without attention and subsequently guide attention towards these stimuli (Berti, 2002; Driver & Vuilleumier, 2001). For instance, in patients with visual extinction, faces with angry or happy expressions are more frequently detected than neutral faces in the contralesional field (Fox, 2002; Vuilleumier & Schwartz, 2001b). Similarly, bodies with emotional gestures are less extinguished than neutral bodies (de Gelder, 2006); and pictures of spiders are less extinguished than pictures of flowers (Vuilleumier & Schwartz, 2001a). Altogether, these results converge with the findings in healthy subjects that emotional stimuli, including facial expressions, may effectively attract attention under conditions of competition with neutral stimuli (Eastwood et al., 2003, Fox et al., 2001). However, to our knowledge, no study has investigated whether such effects of affective facial expression might also bias attention in a visual search task in neglect patients. The fact that emotional cues might alleviate visual extinction in some cases (Fox, 2002; Vuilleumier and Schwartz, 2001a, Vuilleumier and Schwartz, 2001b) does not necessarily predict that similar effects might arise in a visual search task, because search involves some exploratory components that do not contribute to perceptual performance during extinction tests, implies more serial processes in the deployment of attention, and typically entails a much greater load by concomitant distracters (see Husain & Kennard, 1997; Rapcsak, Verfaellie, Fleet, & Heilman, 1989). Moreover, deficits in cancellation tasks and perceptual extinction may dissociate in some patients with neglect (Azouvi et al., 2002; Cocchini, Cubelli, Della Sala, & Beschin, 1999; Fimm et al., 2001; Stone, Halligan, Marshall, & Greenwood, 1998), suggesting that these manifestations have partly different neural substrates (see Karnath, Himmelbach, & Kuker, 2003).

Several studies have previously investigated visual search for neutral targets in neglect, usually to compare parallel/feature and serial/conjunction searches, with a variety of different stimuli and different paradigms (for overview, see Behrmann, Ebert, & Black, 2004). The results have often remained inconclusive. Some studies found evidence for intact feature search in contralesional space (i.e. with intact “pop-out” effects for feature singletons regardless of the number of distracters), suggesting that early “pre-attentive” visual mechanisms extracting these visual features may still operate normally in patients with spatial neglect (Aglioti, Smania, Barbieri, & Corbetta, 1997; Esterman, McGlinchey-Berroth, & Milberg, 2000). Other studies found that search for targets defined by a single feature (e.g., colour, possibly involving parallel “pre-attentive” processes) and search for targets defined by a conjunction of features (e.g., colour + shape, presumably requiring serial attentive processes) both were disrupted in neglect patients (see Behrmann, Ebert et al., 2004; Pavlovskaya, Ring, Groswasser, & Hochstein, 2002; Riddoch & Humphreys, 1987). The latter findings suggest that damage to attentional systems in fronto-parietal networks may impair visual exploration and detection of salient visual attributes in contralesional space during search tasks, even though such attributes may be processed at some pre-attentive stages and produce implicit effects in other tasks (see Berti, 2002). Because faces and expressions are complex visual stimuli defined by specific features as well as a conjunction of features (Maurer, Grand, & Mondloch, 2002; Suzuki & Cavanagh, 1995), and yet may be processed to some extent without attention (Mack & Rock, 1998; Ro, Russell, & Lavie, 2001; Vuilleumier, Sagiv et al., 2001) and even without conscious awareness (de Gelder, Vroomen, Pourtois, & Weiskrantz, 1999; Morris, DeGelder, Weiskrantz, & Dolan, 2001; Pegna, Khateb, Lazeyras, & Seghier, 2005), it is unclear whether search for emotional faces would be more or less preserved than simple feature search in patients with spatial neglect. Our present goal was therefore to test how emotional information modulates the “threshold” for attention and awareness (Duncan & Barrett, 2007; Gaillard et al., 2006)—rather than to manipulate awareness to examine effects on processing of emotional information.

In our study, we asked firstly whether visual search would be guided more efficiently to emotional vs. neutral faces in neglect patients, as previously found in normal subjects (e.g. Eastwood et al., 2003, Fox et al., 2001), despite impaired attention towards contralesional space. Secondly, we also asked whether any deficit in search would dissociate between emotional and simple/low-level feature (colour) cues. Thirdly, we examined whether any emotional advantage during search for faces might differ between negative and positive expressions (i.e., fearful and happy). We designed a simple visual search paradigm in which patients had to report the gender of a target face, whose identity was unique amongst an array of distracters with a different identity. Distracter faces were always neutral. Critically, the target face could differ from the distracters by identity alone (neutral baseline), by identity plus expression (fearful or happy), or by identity plus colour (red hue).

Besides longer latencies for detecting target faces in contralesional/left space, as expected in neglect patients, we predicted two possible outcomes of major theoretical importance: If some “pre-attentive” processing of emotional expression (and/or colour singleton) is intact in neglect patients, and separate from the serial attention processes mediated by fronto-parietal systems, then target faces cued by a different expression (or a different colour) should yield a similar facilitation on both sides of space, with a purely additive gain of these cues over and above any abnormal spatial biases in search (but with no interaction). On the contrary, if visual processes favouring the accuracy of emotional faces (and/or colour singleton) depend on the same attentional systems in fronto-parietal areas that are damaged in neglect patients, then any facilitation for emotional faces (or colour singleton) should be lost in contralesional space, or interact with the pathological disruption of search on that side. Our results support the first hypothesis, by showing that emotional influences on visual search are independent from (but additive to) the mechanisms of spatial attention that are affected by neglect.

Section snippets

Participants

All subjects, healthy and brain-damaged, consented to participate in this study after giving their informed consent according to the local ethics. Healthy participants were 27 volunteers recruited via advertisement at Geneva University and University Hospital. They included 12 males and 15 females, all right-handed (except one left-handed), with a wide age range in order to provide appropriate controls for the patients (range = 22–94, average = 42.9), and had no history of neurological or

Results

We first describe the performance of healthy subjects, for subsequent comparison and interpretation of the results obtained in neglect patients. For each group, we analysed both accuracy rates and latencies of correct responses to the target face, for each of the four critical target conditions.

Intact pattern of cueing effects on visual search in neglect

Our study demonstrates for the first time that, despite right hemisphere damage and abnormal spatial biases in attention, visual search in neglect patients is still reliably influenced by emotional expression in faces as well as by colour cues. In this task, healthy subjects showed significant benefits by task-irrelevant, but distinctive, attributes of faces (colour and expression), indicating that these attributes contributed to make the targets more salient and drew attention more efficiently

Conclusion

The present study has extended previous findings about residual processing of salient visual cues during visual search in neglect patients, and demonstrated that visual search can be modulated by emotional biases, over and above spatial biases in attention caused by right hemisphere damage. Our data provide new answers to current questions concerning interactions between attention and emotion (Phelps, Ling, & Carrasco, 2006; Pessoa et al., 2002, Vuilleumier, 2005). We show that emotional

Acknowledgment

The authors thank Dr. K. N’Diaye for help on low-level stimulus analysis.

This work was supported by a grant from the Swiss National Science Foundation to Prof. Patrik Vuilleumier (632.065935).

References (106)

  • S. Duncan et al.

    The role of the amygdala in visual awareness

    Trends in Cognitive Sciences

    (2007)
  • D. Grandjean et al.

    Effects of emotional prosody on auditory extinction for voices in patients with spatial neglect

    Neuropsychologia

    (2008)
  • S. Hahn et al.

    Aging and visual search: Automatic and controlled attentional bias to threat faces

    Acta Psychologica (Amsterdam)

    (2006)
  • J.V. Haxby et al.

    Human neural systems for face recognition and social communication

    Biological Psychiatry

    (2002)
  • M. Husain et al.

    Distractor-dependent frontal neglect

    Neuropsychologia

    (1997)
  • S. Kastner et al.

    The neural basis of biased competition in human visual cortex

    Neuropsychologia

    (2001)
  • B.J. Liddell et al.

    A direct brainstem–amygdala–cortical ‘alarm’ system for subliminal signals of fear

    Neuroimage

    (2005)
  • A. Mack et al.

    What we see: Inattention and the capture of attention by meaning

    Conscious Cognition

    (2002)
  • D. Maurer et al.

    The many faces of configural processing

    Trends in Cognitive Sciences

    (2002)
  • K. Mogg et al.

    Some methodological issues in assessing attentional biases for threatening faces in anxiety: A replication study using a modified version of the probe detection task

    Behaviour Research and Therapy

    (1999)
  • L. Pessoa et al.

    Attentional control of the processing of neural and emotional stimuli

    Brain Research and Cognitive Brain Research

    (2002)
  • M.I. Posner et al.

    How do the parietal lobes direct covert attention?

    Neuropsychologia

    (1987)
  • G. Pourtois et al.

    Neural systems for orienting attention to the location of threat signals: An event-related fMRI study

    Neuroimage

    (2006)
  • R.D. Rafal

    Neglect

    Current Opinion in Neurobiology

    (1994)
  • D. Sabatinelli et al.

    Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance

    Neuroimage

    (2005)
  • A.M. Treisman et al.

    A feature-integration theory of attention

    Cognitive Psychology

    (1980)
  • G. Vallar

    Extrapersonal visual unilateral spatial neglect and its neuroanatomy

    Neuroimage

    (2001)
  • G. Vallar et al.

    The anatomy of unilateral neglect after right-hemisphere stroke lesions. A clinical/CT-scan correlation study in man

    Neuropsychologia

    (1986)
  • P. Vuilleumier

    How brains beware: Neural mechanisms of emotional attention

    Trends in Cognitive Sciences

    (2005)
  • P. Vuilleumier et al.

    Neural response to emotional faces with and without awareness: Event-related fMRI in a parietal patient with visual extinction and spatial neglect

    Neuropsychologia

    (2002)
  • P. Vuilleumier et al.

    Effects of attention and emotion on face processing in the human brain: An event-related fMRI study

    Neuron

    (2001)
  • P. Vuilleumier et al.

    Explicit and implicit perception of illusory contours in unilateral spatial neglect: Behavioural and anatomical correlates of preattentive grouping mechanisms

    Neuropsychologia

    (2001)
  • P. Azouvi et al.

    Sensitivity of clinical and behavioural tests of spatial neglect after right hemisphere stroke

    Journal of Neurology Neurosurgery and Psychiatry

    (2002)
  • A. Bartels et al.

    The architecture of the colour centre in the human visual brain: New results and a review

    European Journal of Neuroscience

    (2000)
  • S. Bentin et al.

    Controlling interstimulus perceptual variance does not abolish N170 face sensitivity

    Nature Neuroscience

    (2007)
  • A. Berti

    Unconscious processing in neglect

  • Brosch, T., Sander, D., Pourtois, G., & Scherer, K. R. (in press). Beyond fear: Rapid spatial orienting toward positive...
  • T. Brosch et al.

    That baby caught my eye… attention capture by infant faces

    Emotion

    (2007)
  • C. Cavada et al.

    The anatomical connections of the macaque monkey orbitofrontal cortex. A review

    Cerebral Cortex

    (2000)
  • B. de Gelder

    Towards the neurobiology of emotional body language

    Nature Review of Neuroscience

    (2006)
  • B. de Gelder et al.

    Non-conscious recognition of affect in the absence of striate cortex

    Neuroreport

    (1999)
  • J. Duncan et al.

    Visual search and stimulus similarity

    Psychological Review

    (1989)
  • J.D. Eastwood et al.

    Negative facial expression captures attention and disrupts performance

    Perception & Psychophysics

    (2003)
  • M. Esterman et al.

    Preattentive and attentive visual search in individuals with hemispatial neglect

    Neuropsychology

    (2000)
  • M.J. Fenske et al.

    Modulation of focused attention by faces expressing emotion: Evidence from flanker tasks

    Emotion

    (2003)
  • B. Fimm et al.

    Asymmetries of visual attention after circumscribed subcortical vascular lesions

    Journal of Neurology Neurosurgery and Psychiatry

    (2001)
  • E. Fox

    Processing emotional facial expressions: The role of anxiety and awareness

    Cognition Affective Behavioural Neuroscience

    (2002)
  • E. Fox et al.

    Do threatening stimuli draw or hold visual attention in subclinical anxiety?

    Journal of Experimental Psychology-General

    (2001)
  • E. Fox et al.

    Anxiety modulates the degree of attentive resources required to process emotional faces

    Cognition Affective Behavioural Neuroscience

    (2005)
  • R. Gaillard et al.

    Nonconscious semantic processing of emotional words modulates conscious access

    Proceedings of the National Academy of Sciences of the United States of America

    (2006)
  • Cited by (47)

    • Amygdala function in emotion, cognition, and behavior

      2022, Handbook of Clinical Neurology
    View all citing articles on Scopus
    View full text