Elsevier

Current Opinion in Neurobiology

Volume 40, October 2016, Pages 94-102
Current Opinion in Neurobiology

From the optic tectum to the primary visual cortex: migration through evolution of the saliency map for exogenous attentional guidance

https://doi.org/10.1016/j.conb.2016.06.017Get rights and content

Highlights

  • A saliency map in the primary visual cortex for primates.

  • A saliency map in the optic tectum for archer fish.

  • Through evolution, the saliency map migrated from the optic tectum to the primary visual cortex.

Recent data have supported the hypothesis that, in primates, the primary visual cortex (V1) creates a saliency map from visual input. The exogenous guidance of attention is then realized by means of monosynaptic projections to the superior colliculus, which can select the most salient location as the target of a gaze shift. V1 is less prominent, or is even absent in lower vertebrates such as fish; whereas the superior colliculus, called optic tectum in lower vertebrates, also receives retinal input. I review the literature and propose that the saliency map has migrated from the tectum to V1 over evolution. In addition, attentional benefits manifested as cueing effects in humans should also be present in lower vertebrates.

Introduction

The saliency of a visual location, at least as we define it here, is the degree to which this location attracts attention or gaze exogenously. Exogenous guidance is reflexive or bottom-up, driven by external rather than internal factors (such as the goal of an on-going task), which are endogenous or top-down. For example, the location of an orientation singleton, such as a vertical bar in a background of horizontal bars, is salient or has a high saliency value; so is the location of a color singleton, such as a red dot among many green ones. Such feature singletons, examples of which are shown in Figure 1a, are so salient that they are said to pop-out perceptually. Saliency at a location in an image can be measured by the brevity of the reaction time (RT) taken to saccade towards, or find a target at, this location. In humans and monkeys, guiding attention exogenously to a spatial location, such as by flashing a brief cue at this location, increases the speed and accuracy of detecting, recognizing, or discriminating visual inputs appearing subsequently at the cued location [1•, 2]. This attentional benefit is called the cueing effect and can also be used to measure the saliency of the cue.

Traditional views [3] presume that it is higher brain areas, such as the frontal eye field (FEF) in humans, that contain a saliency map of the visual world to guide attention exogenously. This was partly inspired by the observations that saliency is a general property that could arise from visual inputs with any feature values (e.g., vertical or red) in any feature dimension (e.g., color, orientation, and motion) whereas neurons in lower visual areas such as the primary visual cortex (V1) are tuned to specific feature values (e.g., the vertical orientation) rather than being feature untuned. However, recent behavioral data in humans, combined with physiological knowledge, support the hypothesis that V1 creates the saliency map [4, 5••]. Meanwhile, FEF is often considered to be responsible for the different function of endogenous attentional guidance.

Although the idea of computing saliency without frontal/parietal brain areas contradicts traditional wisdom, it is perhaps more appealing when considering lower mammals such as rats or non-mammalian vertebrates such as fish. All vertebrates should have visual attentional mechanisms, as attention helps to focus brain's limited processing power to a fraction of visual inputs; however, prefrontal cortical areas (which include FEF) comprise a smaller fraction of the whole cortex [6, 7] in lower mammals, and fish lack neocortex. For such lower vertebrates, it appears that the superior colliculus (SC) (called optic tectum (OT) in non-mammals) is involved in visual (attentional) orienting [8].

Note that saliency at a location of the same visual item is context dependent, for example, a vertical bar is salient among many horizontal bars but not among other identical vertical bars. Hence building a saliency map requires analyzing visual features (such as color, orientation, and motion direction) and comparing features at different visual locations. Figure 2a outlines the essential building blocks for generating and utilizing the saliency map in vision. To generate the map, some degree of visual analysis is necessary to, first, build feature tuning in neurons and to, second, compute saliency by additional processing of the responses from feature tuned neurons, effectively comparing features at different visual locations. For the saliency map to impact behavior, there should be (1) a winner-take-all (WTA) computation over the saliency map to identify the most salient location (when the saliency map is combined with endogenous factors and other sensory input, the net outcome is what is called a priority map [10, 11], and WTA will then identify the location of the highest priority); and (2) either overt or covert orienting towards the winning location (for simplicity, we omit avoidance behavior). In overt orienting, the attended location is shifted to the center of the visual field or is acted upon/towards (e.g., in predatory behavior). Both covert and overt orienting impact visual analysis by, for example, gain controls to enhance neural responses or sensitivities [2, 11, 12], to lead to the cueing effects.

The location of the WTA computation appears to be conserved over evolution. Across vertebrates [7, 13••], SC/OT is retinotopic. Its upper layers receive visual inputs (from the retina and forebrain/V1); its intermediate and deep layers receive inputs from the upper layers and receive context inputs, which include (spatially aligned) sensory inputs of non-visual modalities and other inputs from the forebrain (such as endogenous inputs from FEF in primates) [7, 13••]. WTA appears to occur in the intermediate and deep layers of SC/OT, activities at different visual locations suppress each other so that the response to the winner location is higher than the responses to the other locations [7, 13••, 14••, 15•, 16•, 17••, 18]. Along with orienting to the attended location (e.g., by directing a saccade), the outcome of WTA can impact visual analysis in at least two further ways via efferent projections from SC/OT. One of these innervates dorsal thalamic regions (such as pulvinar and lateral geniculate nucleus (LGN)), which in turn project to cortex or to forebrain areas in non-mammals. The second efferent projection is direct to the retina in lower vertebrates [7].

This paper argues that the brain region that realizes the saliency map is not conserved across species. I will review recent findings which suggest that V1 could realize a saliency map which guides attention exogenously in primates, and propose that, over the course of evolution, the saliency map migrated from the OT/SC to V1, just as much visual analysis migrated from subcortical areas to the cortex.

To avoid confusion, we must first clarify what is meant by the claim that a saliency map arises in a particular brain region. This is that the saliency values are first computed and explicitly represented in neural responses in this brain region. Downstream brain regions can inherit the saliency values for further processing or action. For example, when this map is first created in V1, it can be projected directly or indirectly to SC or FEF. In turn, they can read out the saliency values to execute a shift in attention or arrange for exogenous and endogenous influences over attention to compete or be combined. Although the read-out makes a copy of the saliency map in downstream areas, these downstream areas are not deemed responsible for the saliency map, just as a printer attached to a computer is not responsible for any graphics sent from the computer for printing. SC neurons in monkeys are normally untuned to any visual feature [13••], making the monkey SC a relatively pure map of space that is devoid of visual feature analysis. Of course, such a pure space map is appropriate for performing a WTA operation on a saliency map read out from an upstream source.

Section snippets

A saliency map in V1 for primates

Noting that the activities of V1 neurons could serve as a universal currency to bid for attention regardless of those neurons’ preferred features, Li [4, 5••] proposed that the saliency map is created in V1. According to this, the saliency at any location in a scene is the highest V1 response to input at that location relative to the highest responses to other locations, see Figure 1c. The neural mechanism by which V1 responds more vigorously to salient feature singletons than to the

Concluding remarks

In summary, it appears that through the evolutionary expansion of the forebrain, visual analysis increasingly migrated from the optic tectum to the forebrain, in particular to V1 [13••]. This was accompanied by decreases in the percentage of retinal ganglion cells that project to the tectum. It was also accompanied by increases in the percentage of retinal ganglion cells which, like the midget cells in monkey, are not special feature detectors, but rather encode visual information efficiently

Conflict of interest statement

Nothing declared.

References and recommended reading

Papers of particular interest, published within the period of review, have been highlighted as:

  • • of special interest

  • •• of outstanding interest

Acknowledgement

This work is supported by the Gatsby Charitable Foundation, and I would like to thank Tom Baden, David Berson, Onkar Dhande, Marcus Meister, Ronen Segev, and Ben Sivyer for helping with the literature, and Peter Dayan for comments on the paper.

References (70)

  • Z. Li

    Contextual influences in V1 as a basis for pop out and asymmetry in visual search

    Proc Natl Acad Sci U S A

    (1999)
  • Z. Li

    A saliency map in primary visual cortex

    Trends Cogn Sci

    (2002)
  • A.B. Butler et al.

    Comparative Vertebrate Neuroanatomy: Evolution and Anatomy

    (2005)
  • B.J. White et al.

    The superior colliculus

  • L. Zhaoping et al.

    Primary visual cortex as a saliency map: a parameter-free prediction and its test by behavioral data

    PLoS Comput Biol

    (2015)
  • J. Bisley et al.

    Attention, intention, and priority in the parietal lobe

    Annu Rev Neurosci

    (2011)
  • L. Zhaoping

    Understanding Vision: Theory, Models, and Data

    (2014)
  • F. Wang et al.

    Modulation of neuronal responses by exogenous attention in macaque primary visual cortex

    J Neurosci

    (2015)
  • P.H. Schiller

    The superior colliculus and visual function

  • D.P. Munoz et al.

    Lateral inhibitory interactions in the intermediate layers of the monkey superior colliculus

    J Neurophysiol

    (1998)
  • R.M. McPeek et al.

    Saccade target selection in the superior colliculus during a visual search task

    J Neurophysiol

    (2002)
  • S.P. Mysore et al.

    Signaling of the strongest stimulus in the owl optic tectum

    J Neurosci

    (2011)
  • E.I. Knudsen

    Control from below: the role of a midbrain network in spatial attention

    Eur J Neurosci

    (2011)
  • C.R. Vokoun et al.

    Intralaminar and interlaminar activity within the rodent superior colliculus visualized with voltage imaging

    J Neurosci

    (2010)
  • J. Allman et al.

    Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local–global comparisons in visual neurons

    Annu Rev Neurosci

    (1985)
  • L. Zhaoping

    Attention capture by eye of origin singletons even without awareness  a hallmark of a bottom-up saliency map in the primary visual cortex

    J Vis

    (2008)
  • L. Zhaoping

    Gaze capture by eye-of-origin singletons: Interdependence with awareness

    J Vis

    (2012)
  • D.H. Hubel et al.

    Ferrier lecture: functional architecture of macaque monkey visual cortex

    Proc R Soc Lond Ser B Biol Sci

    (1977)
  • A. Burkhalter et al.

    Processing of color, form and disparity information in visual areas vp and V2 of ventral extrastriate cortex in the macaque monkey

    J Neurosci

    (1986)
  • T. Lock et al.

    Distribution of corticotectal cells in macaque

    Exp Brain Res

    (2003)
  • C.M. Cerkevich et al.

    Distribution of cortical neurons projecting to the superior colliculus in macaque monkeys

    Eye Brain

    (2014)
  • P. Schiller

    The neural control of visually guided eye movements

  • P.H. Schiller et al.

    The effect of frontal eye field and superior colliculus lesions on saccadic latencies in the rhesus monkey

    J Neurophysiol

    (1987)
  • C. Bruce et al.

    The primate frontal eye field

  • T. Isa et al.

    Saccade control after V1 lesion revisited

    Curr Opin Neurobiol

    (2009)
  • Cited by (50)

    View all citing articles on Scopus
    View full text