The sensitivity of primate STS neurons to walking sequences and to the degree of articulation in static images
Introduction
Artists use many tricks to convey information about movement. One method commonly used is to illustrate a person with legs and arms outstretched or articulated as if the artist had captured a snapshot of the person mid-stride during walking or running. When we see such static images we commonly interpret the human as moving, walking or running forwards through the scene. Although no real movement occurs, the articulated human figure ‘implies’ movement forward by its configuration or form. There is considerable evolutionary advantage in this ability to infer information about movement from the posture; we can interpret movement direction and speed from a momentary glimpse of a figure.
Traditionally, form and motion information have been thought to be processed along anatomically separate pathways; relatively little effort has been spent investigating how the pathways interact and how motion and form are integrated. Recently, however, three fMRI studies have shown that the brain structure that processes motion, hMT+/V5 (Zeki et al., 1991; Watson et al., 1993; Tootell et al., 1995), is more active to images implying motion when compared to similar images where motion in not implied (Kourtzi and Kanwisher, 2000; Senior et al., 2000; Krekelberg et al., 2005). In each study very different images were used to imply motion; Kourtzi and Kanwisher used images of athletes and animals in action, Senior et al. used images of moving objects and Krekelberg et al. used ‘glass patterns’, i.e., arrangements of dots suggesting a path of motion. These papers all argue that information regarding the form of static images is made available to hMT+/V5 for coding motion.
Neurons in the monkey homologue of human hMT+/V5, the medial temporal (MT) and medial superior temporal (MST) areas, also respond to glass patterns, where motion is implied (Krekelberg et al., 2003). Areas MT and MST contain neurons that respond to motion (Dubner and Zeki, 1971; Desimone and Ungerleider, 1986) and respond in correlation with the monkey's perception of motion (Newsome et al., 1986; Newsome and Pare, 1988). Neurons in MT/MST area respond maximally to movement in one direction; Krekelberg et al. (2003) showed that they respond preferentially to both real dot motion and implied motion in the preferred direction. Presentation of contradictory implied motion and real motion results in a compromised MT/MST neural response and compromises the monkey's perception of coherent movement.
The blood-oxygen level-dependent (BOLD) activity seen in human hMT+/V5 to complex images implying motion (Kourtzi and Kanwisher, 2000; Senior et al., 2000) could be explained by input from other regions of the cortex. Measurement of event-related potentials (ERP) responses from a dipole pair in the occipital lobe, consistent with localization to hMT+/V5, showed that the responses to the real motion of a random-dot field were 100 ms earlier than responses to static images containing human figures implying motion (Lorteije et al., 2006). The delay in the implied motion response indicates that this information arrives via a different and longer pathway. Kourtzi and Kanwisher (2000) concluded that since inferring information about still images depends upon categorization and knowledge, this must be analysed elsewhere. The activation of hMT+/V5 by implied motion of body images could be due to top-down influences. Senior et al. (2000) suggested that the activation they saw in hMT+/V5 is more likely due to processing of the form of the image in temporal cortex without the need for engagement of conceptual knowledge. At present, there is no evidence that cells in monkey MT are sensitive to articulated human figures implying motion despite active search (Jeanette Lorteije, personal communication).
Information about body posture and articulation in a human figure is likely to come from regions of the cortex that contain neurons sensitive to body form. The superior temporal sulcus (STS) in monkeys and the superior temporal gyrus (STG) and nearby cortex in humans is widely believed to be responsible for processing socially important information. Monkey STS contains neurons that respond to movement of human bodies (Bruce et al., 1981; Perrett et al., 1985), the form (view) of human bodies (Wachsmuth et al., 1994) and many appear to integrate motion and form to code walking direction (Oram and Perrett, 1996; Jellema et al., 2004). It is not known, however, if cells exist that are sensitive to the pattern of articulation that may differentiate postures associated with motion from those associated with standing still.
Giese and Poggio (2003) extended models of object recognition (Riesenhuber and Poggio, 1999, Riesenhuber and Poggio, 2002) to generate a plausible feed-forward model of biological motion recognition. A critical postulate of Giese and Poggio's model is the existence of ‘snapshot’ neurons, neurons tuned to differing degrees of articulation of bodies. Giese and Poggio suggest that these neurons should be found in inferotemporal (IT) or STS cortex, and would feed-forward to neurons coding specific motion patterns, e.g., walking (Oram and Perrett, 1996; Jellema et al., 2004).
In this study we set out to investigate if neurons in temporal cortex can code the degree of articulation of a human figure. Video taping a person walking or running produces a series of stills capturing discrete moments in time. Some of these stills show the person in an articulated pose, others in less-articulated poses akin to standing still. We made use of such video footage in order to compare the responses of STS neurons to a human figure articulated and standing. Neurons in STS sensitive to non-walking articulated postures are also sensitive to actions leading to such postures (Jellema and Perrett, 2003). It is possible, however, to arrive at a posture from two different directions, by walking forwards, or by walking backwards, both movement directions are consistent with the same static form. We therefore used the video footage played forwards and backwards to investigate how form sensitivity was related to walking.
Following Giese and Poggio (2003) we hypothesized that STS neurons would discriminate articulated postures from standing postures. We also hypothesized that the ability to differentiate posture in static images would relate to sensitivity to motion type for the same neurons. To this end we explore the cells’ sensitivity to images of static figures taken from video and movies containing the same images, played forward and in reverse. We also investigate the sensitivity to body view since cells sensitive to static and moving bodies exhibit viewpoint sensitivity (Perrett et al., 1991; Oram and Perrett, 1996).
Section snippets
Physiological subjects, recording and reconstruction techniques
One rhesus macaque, aged 9 years, was trained to sit in a primate chair with head restraint. Using standard techniques (Perrett et al., 1985), recording chambers were implanted over both hemispheres to enable electrode penetrations to reach the STS. Cells were recorded using tungsten microelectrodes inserted through the dura mater. The subject's eye position (±1°) was monitored (IView, SMI, Germany). A Pentium IV PC with a Cambridge electronics CED 1401 interface running Spike 2 recorded eye
Results
We tested 55 cells that responded significantly (see methods) to either static images of a human figure or movies of a human walking.
Discussion
The results of this study show two main findings: (1) Fifty-seven per cent of STS neurons that respond to static images of a human figure are sensitive to the degree of articulation of the figure itself. (2) There is an association between STS neuronal response sensitivity to the degree of articulation of a human figure and sensitivity to the compatibility between the direction of locomotion and view of the body of a human walking. For the cells that were sensitive to both the degree of
Acknowledgments
This work was funded by grants from the EU and the Wellcome Trust.
References (40)
- et al.
Visual areas in the temporal cortex of the macaque
Brain Res.
(1979) - et al.
Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey
Brain Res.
(1971) - et al.
Cells in monkey STS responsive to articulated body motions and consequent static posture: a case of implied motion
Neuropsychologia
(2003) - et al.
Visual analysis of body movements by neurons in the temporal cortex of the macaque monkey: a preliminary report
Behav. Brain Res.
(1985) - et al.
Neural mechanisms of object recognition
Curr. Opin. Neurobiol.
(2002) - et al.
The functional neuroanatomy of implicit-motion perception ‘representational momentum’
Curr. Biol.
(2000) Aligning pictorial descriptions: An approach to object recognition
Cognition
(1989)- et al.
Integration of visual and auditory information by STS neurons responsive to the sight of actions
J. Cogn. Neurosci.
(2005) - et al.
Functional subdivisions of the temporal neocortex
J. Neurosci.
(1987) - et al.
Perception of biological motion without local image motion
Proc. Natl. Acad. Sci. USA
(2002)
Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque
J. Neurophysiol.
Both striate cortex and superior colliculus contribute to visual properties of neurons in superior temporal polysensory area of macaque monkey
J. Neurophysiol.
Multiple visual areas in the caudal superior temporal sulcus of the macaque
J. Comp. Neurol.
Cortical connections of inferior temporal area TEO in macaque monkeys
J. Comp. Neurol.
Color sensitivity of cells responsive to complex stimuli in the temporal cortex
J. Neurophysiol.
Rapid serial visual presentation for the determination of neural selectivity in area STSa
Prog. Brain Res.
Neural mechanisms for the recognition of biological movements
Nat. Rev. Neurosci.
Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey
J. Neurophysiol.
Single cell integration of animate form, motion and location in the superior temporal cortex of the macaque monkey
Cereb. Cortex
Visual perception of biological motion and a model for its analysis
Percept. Psychophys.
Cited by (67)
Categorization learning induced changes in action representations in the macaque STS
2023, NeuroImageCitation Excerpt :Some of the first evidence suggesting the role of the STS in representing observed body parts and actions was already obtained more than four decades ago. Since this early demonstration of STS neurons showing responses to a walking person (Bruce et al., 1981), subsequent single cell studies showed that the macaque STS contained neurons in both banks and fundus responding to a range of stimuli involving bodies or body parts, either presented statically or in motion (Perrett et al., 1989; Oram and Perrett, 1996; Barraclough et al., 2006; Jellema and Perrett, 2006; Vangeneugden et al., 2009; Singer and Sheinberg, 2010). In line with these previous reports, our findings suggest observation of dynamic actions yield responses throughout a large portion of the lower bank and fundus, including posterior MT and FST, in addition to middle and anterior body patches.
Evidence for a Third Visual Pathway Specialized for Social Perception
2021, Trends in Cognitive SciencesInteroceptive signals impact visual processing: Cardiac modulation of visual body perception
2017, NeuroImageCitation Excerpt :Thus, a first activation occurs during the categorization of the bodily image (at 100 ms: P1) (de Gelder et al., 2010), followed by activations that are associated with a more refined structural body-specific processing, occurring between 150 and 250 ms after stimulus onset (N190) (Thierry et al., 2006; Ishizu et al., 2010). These activations are found within the temporo-occipital cortex, likely overlapping with the extrastriate body area (see Barraclough et al., 2006; Pourtois et al., 2007). Although several studies have supported the role of the extrastriate body area in multisensory (e.g., visuo-auditory) integration (Beer et al., 2013; Limanowski and Blankenburg, 2016), it is debated if this area is a unimodal visual area or an important region for multisensory bodily processing (Downing and Peelen, 2011).
Body Perception
2015, Brain Mapping: An Encyclopedic ReferenceDynamics of walking adaptation aftereffects induced in static images of walking actors
2012, Vision ResearchCitation Excerpt :Again, results from monkey single unit recording may provide an answer. For cells coding walking actions there is a relationship between the selectivity for the walking action and selectivity for different static images depicting walking postures (Barraclough et al., 2006). Cells that code forward walking respond significantly more to static images of walkers in articulated postures, cells that code backward walking respond significantly more to static images in standing postures.