Elsevier

NeuroImage

Volume 155, 15 July 2017, Pages 437-449
NeuroImage

Task alters category representations in prefrontal but not high-level visual cortex

https://doi.org/10.1016/j.neuroimage.2017.03.062Get rights and content

Abstract

A central question in neuroscience is how cognitive tasks affect category representations across the human brain. Regions in lateral occipito-temporal cortex (LOTC), ventral temporal cortex (VTC), and ventro-lateral prefrontal cortex (VLFPC) constitute the extended “what” pathway, which is considered instrumental for visual category processing. However, it is unknown (1) whether distributed responses across LOTC, VTC, and VLPFC explicitly represent category, task, or some combination of both, and (2) in what way representations across these subdivisions of the extended ‘what’ pathway may differ. To fill these gaps in knowledge, we scanned 12 participants using fMRI to test the effect of category and task on distributed responses across LOTC, VTC, and VLPFC. Results reveal that task and category modulate responses in both high-level visual regions, as well as prefrontal cortex. However, we found fundamentally different types of representations across the brain. Distributed responses in high-level visual regions are more strongly driven by category than task, and exhibit task-independent category representations. In contrast, distributed responses in prefrontal cortex are more strongly driven by task than category, and contain task-dependent category representations. Together, these findings of differential representations across the brain support a new idea that LOTC and VTC maintain stable category representations allowing efficient processing of visual information, while prefrontal cortex contains flexible representations in which category information may emerge only when relevant to the task.

Introduction

The human ventral stream, also referred to as the “what” pathway, extends from occipital cortex to high-level visual regions in lateral occipito-temporal cortex (LOTC) and ventral temporal cortex (VTC), and is involved in object recognition and categorization (Goodale and Milner, 1992, Grill-Spector and Weiner, 2014, Mishkin and Ungerleider, 1982). LOTC and VTC have both clustered (Cohen et al., 2000, Downing et al., 2001, Epstein and Kanwisher, 1998; Kanwisher, 1997) and distributed representations of object categories (Grill-Spector and Weiner, 2014, Haxby et al., 2001), which are organizational features believed to contribute to the speed of categorization (Thorpe et al., 1996). Further, a large body of research has documented that distributed responses across LOTC and VTC contain linearly separable category representations (Cox and Savoy, 2003, Grill-Spector and Weiner, 2014, Jacques et al., 2016, Kravitz et al., 2011, Kriegeskorte et al., 2008, Sayres and Grill-Spector, 2008, Spiridon and Kanwisher, 2002, Walther et al., 2009, Weiner and Grill-Spector, 2010). Linear separability is computationally advantageous because it enables combinatorial coding of distributed responses resulting in representations of tens of thousands of categories (Haxby et al., 2001, Haxby et al., 2011), as well as rapid and efficient readout of category information (DiCarlo and Cox, 2007, Grill-Spector and Weiner, 2014).

The “what” pathway is thought to extend from VTC to ventro-lateral prefrontal cortex (VLPFC) for two reasons. First, VTC is anatomically connected to VLPFC via white matter connections such as the uncinate fasciculus (Catani et al., 2002, De Schotten et al., 2012, Gerbella et al., 2010, Schmahmann and Pandya, 2006, Ungerleider et al., 1989). Second, VLPFC is sensitive to the content of visual stimuli, rather than the location of visual stimuli, particularly under selective attention (Baldauf and Desimone, 2014, Çukur et al., 2013, Harel et al., 2014, Peelen et al., 2009, Rushworth et al., 2005) and working memory tasks (Courtney et al., 1997; Goldman-Rakic, 1996; Muller et al., 2002; Volle et al., 2008; Demb et al., 1995). Additional evidence suggests that “what” responses related to stimulus attributes emerge in VLPFC only when they are task-relevant (Asaad et al., 2000, Johnston and Everling, 2006, Lee and Baker, 2016, McKee et al., 2014; Miller and Cohen, 2000; Waskom et al., 2014). Thus, while VLPFC is considered part of the extended “what” pathway (Kravitz et al., 2013, Romanski, 2012, Wilson et al., 1993), VLPFC neural responses likely do not represent visual attributes alone (Rigotti et al., 2013, Romanski, 2004).

Since a key computational goal of the ventral “what” pathway is categorization (Haxby et al., 2001, Kiani et al., 2007, Kriegeskorte et al., 2008, Grill-Spector and Weiner, 2014), it is natural to ask whether VLPFC responses also contain category representations. Previous studies provide some evidence for visual category representations in VLPFC. Indeed, both electrophysiology experiments in macaques and fMRI measurements in humans observed face-selective responses (Scalaidhe et al., 1997, Tsao et al., 2008, Chan, 2013), as well as representation of categorical boundaries between pairs of stimuli (Freedman et al., 2001, Miller et al., 2002, Jiang et al., 2007, Meyer et al., 2011). For example, Jiang et al. (2007) showed that human VLPFC responses are sensitive to categorical boundaries, rather than physical differences among exemplars of the category. However, the nature of distributed responses in VLPFC has not been extensively investigated. Thus, it remains unknown if distributed responses in VLPFC contain category representations for multiple categories, and if these representations are robust and linearly separable as found in high-level visual cortex.

Notably, research in both human and non-human primates suggests that category-selective responses in VLPFC may be task-dependent (Jiang et al., 2007, McKee et al., 2014, Rigotti et al., 2013, Romanski, 2004). For example, McKee et al. (2014) reported that category-selective responses in VLPFC were observed only when category distinctions were task-relevant, but not during passive viewing. Additionally, researchers have observed that selective attention to categories alters distributed responses in VLPFC (Çukur et al., 2013; Peleen et al., 2009). However, prior studies differ on how task and attention affect category-selective responses in high-level visual cortex. While some studies report task independent category-selective responses (McKee et al., 2014, Jiang et al., 2007), other studies indicate that selective attention can alter distributed responses in high-level visual cortex (Çukur et al., 2013; Peleen et al., 2009). Thus, it remains unresolved whether task effects are equivalent or different from the effects of selective attention, and if these effects are region-specific or common across the extended “what” pathway.

Given these unresolved issues, we asked: (1) Do distributed responses across LOTC, VTC, and VLPFC explicitly represent category, task, or some combination of both? (2) Do category representations differ across cortical regions constituting the extended “what” pathway? (3) Are category representations similarly affected by task and selective attention?

To address these questions, we measured distributed responses in LOTC, VTC, and VLPFC while participants viewed images from five object categories and performed three types of cognitive tasks: oddball (OB), working memory (WM), and selective attention (SA, Fig. 1). We chose these tasks as they (1) have been widely used in studies of high-level visual cortex and VLPFC, (2) vary in the cognitive operations required, and (3) vary in the relevancy of the stimulus to the task. We first investigated how task and visual category affect amplitudes of distributed responses across the extended “what” pathway. We then examined how task and selective attention affect distributed category-selective representations. Critically, for each of these analyses, we examined whether effects were uniform or differential across LOTC, VTC, and VLPFC.

Section snippets

Participants

Twelve subjects (5 female, ages 23–44) from Stanford University participated in three experiments. All subjects were right-handed and had normal or corrected-to-normal vision. Participants gave written informed consent. All procedures were approved by the Stanford Internal Review Board on Human Subjects Research.

Stimuli

In all experiments, subjects viewed grayscale images of stimuli from five categories: faces, bodies, cars, houses, and pseudowords (Fig. 1A, example stimuli). We chose these categories

Data acquisition

Subjects were scanned using a General Electric Sigma MR750 3T scanner located in the Center for Cognitive and Neurobiological Imaging (CNI) at Stanford University using a custom-built 32-channel head coil. Using an EPI sequence with level 3 multi-slice acceleration, we acquired 48 slices at 2.4 mm isotropic resolution, FOV=192 mm, TE=30 ms, TR=1 s, and flip angle=62°. The slice prescription covered the entire brain, except the very superior portion of the cortex, roughly corresponding to superior

How do task and visual category affect distributed responses in LOTC, VTC, and VLPFC?

To address this question, we examined the information in distributed multivoxel patterns (MVPs) of response amplitudes across VTC, LOTC, and VLPFC for each of the five visual categories and the three cognitive tasks. To test the relationship among MVPs for these 15 experimental conditions, we generated representational similarity matrices (RSMs) for each anatomical ROI within each subject. Each cell in the RSM indicates the similarity (Pearson correlation) between a pair of MVPs for a specific

Discussion

In the present study, we measured distributed responses in the extended “what” pathway including high-level visual cortex (LOTC and VTC) and prefrontal cortex (VLPFC) while participants viewed exemplars from multiple categories and performed different tasks. Our results revealed four main findings. First, the amplitudes of distributed responses throughout the extended “what” pathway are driven by both visual category and cognitive task. Second, distributed responses in high-level visual cortex

Conclusions

In sum, we suggest that differential representations of category and task across high-level visual and prefrontal cortices reflect the brain's need for a balance between stable and flexible category representations to accommodate task goals. Too much stability would generate rigid neural representations unable to accommodate new information, while too much flexibility would generate a confusable and unstable representation of the visual world that is functionally unreliable. The present data

Acknowledgements

This work was supported by the National Eye institute (NEI) Grant 1R01EY02391501A1 and the Stanford CNI seed grant.

References (79)

  • X. Jiang et al.

    Evaluation of a shape-based model of human face discrimination using fMRI and behavioral techniques

    Neuron

    (2006)
  • D.J. Kravitz et al.

    The ventral visual pathway: an expanded neural framework for the processing of object quality

    Trends Cogn. Sci.

    (2013)
  • N. Kriegeskorte et al.

    Matching categorical object representations in inferior temporal cortex of mand and monkey

    Neuron

    (2008)
  • M. Mishkin et al.

    Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys

    Behav. Brain Res.

    (1982)
  • R. Saxe et al.

    Divide and conquer: a defense of functional localizers

    Neuroimage

    (2006)
  • M. Spiridon et al.

    How distributed is visual category information in human occipito-temporal cortex? An fMRI study

    Neuron

    (2002)
  • K.S. Weiner et al.

    Sparsely-distributed organization of face and limb activations in human ventral temporal cortex

    Neuroimage

    (2010)
  • K.S. Weiner et al.

    Not one extrastriate body area: using anatomical landmarks, hMT+, and visual field maps to parcellate limb-selective activations in human lateral occipitotemporal cortex

    Neuroimage

    (2011)
  • K.S. Weiner et al.

    The mid-fusiform sulcus: a landmark identifying both cytoarchitectonic and functional divisions of human ventral temporal cortex

    Neuroimage

    (2014)
  • W.F. Asaad et al.

    Task-specific neural activity in the primate prefrontal cortex

    J. Neurophysiol.

    (2000)
  • D. Baldauf et al.

    Neural mechanisms of object-based attention

    Science

    (2014)
  • S. Bracci et al.

    Task context overrules object- and category-related representational content in the human parietal cortex

    Cereb. Cortex

    (2017)
  • A.W.-Y. Chan

    Functional organization and visual representations of human ventral lateral prefrontal cortex

    Front. Psychol.

    (2013)
  • L. Cohen et al.

    The visual word form area: spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients

    Brain

    (2000)
  • S. Courtney et al.

    Transient and sustained activity in a distributed neural system for human working memory

    Nature

    (1997)
  • T. Çukur et al.

    Attention during natural vision warps semantic representation across the human brain

    Nat. Neurosci.

    (2013)
  • B.M. Crittenden et al.

    Task difficulty manipulation reveals multiple demand activity but no frontal lobe hierarchy

    Cereb. Cortex

    (2014)
  • M.T. De Schotten et al.

    Monkey to human comparative anatomy of the frontal lobe association tracts

    Cortex

    (2012)
  • J.B. Demb et al.

    Semantic encoding and retrieval in the left inferior prefrontal cortex: a functional MRI study of task difficulty and process specificity

    J. Neurosci.

    (1995)
  • P.E. Downing et al.

    A cortical area selective for visual processing of the human body

    Science

    (2001)
  • J. Dubois et al.

    Single-unit recordings in the macaque face patch system reveal limitations of fMRI MVPA

    J. Neurosci.: Off. J. Soc. Neurosci.

    (2015)
  • R. Epstein et al.

    A cortical representation of the local visual environment

    Nature

    (1998)
  • D.J. Freedman et al.

    Categorical representation of visual stimuli in the primate prefrontal

    Cortex

    (2001)
  • D.J. Freedman et al.

    A comparison of primate prefrontal and inferior temporal cortices during visual categorization

    J. Neurosci.: Off. J. Soc. Neurosci.

    (2003)
  • Gerbella, M., Belmalih, A., Borra, E., Rozzi, S., Luppino, G., 2010. Cortical Connections of the Macaque Caudal...
  • I. Gauthier et al.

    Expertise for cars and birds recruits brain areas involved in face recognition

    Nat. Neurosci.

    (2000)
  • G. Golarai et al.

    Differential development of high-level visual cortex correlates with category-specific recognition memory

    Nat. Neurosci.

    (2007)
  • P.S. Goldman-Rakic

    Regional and cellular fractionation of working memory

    Proc. Natl. Acad. Sci. USA

    (1996)
  • K. Grill-Spector et al.

    The functional architecture of the ventral temporal cortex and its role in categorization

    Nat. Rev. Neurosci.

    (2014)
  • Cited by (37)

    • Attention enhances category representations across the brain with strengthened residual correlations to ventral temporal cortex

      2022, NeuroImage
      Citation Excerpt :

      It is possible that if we had used a different attention task, such as a spatial attention task, we may have found larger attentional effects in the parietal lobe, in line with previous spatial attention results (Peelen et al., 2009; Sprague & Serences, 2013). These results also extend prior findings from our lab showing that both attended and unattended category information can be decoded from distributed responses across the entire lateral occipitotemporal complex (LOTC) as well as ventral temporal cortex (VTC), but that only attended information can be decoded from distributed responses across the entire ventrolateral prefrontal cortex (VLPFC) (Bugatus et al., 2017). Extending our prior results, we not only show the effects of selective attention to attended vs ignored stimuli relative to training with independent data using a different (oddball) task but also reveal the effect of attention at a finer spatial resolution across the entire brain.

    View all citing articles on Scopus
    View full text