Non-identical neural mechanisms for two types of mental transformation: event-related potentials during mental rotation and mental paper folding

https://doi.org/10.1016/S0028-3932(03)00060-5Get rights and content

Abstract

Reaction times, accuracy and 128-channel event-related potentials (ERPs) were measured from 14 normal, right-handed subjects while they performed two different parity-judgment tasks that require transformations of mental images: a relatively simple task requiring a single transformation (mental letter rotation), and a more complex task involving a coordinated sequence of transformations (mental paper folding). Reaction times increased monotonically with larger angular displacements from the upright (for mental rotation) and with number of squares carried (for mental paper folding). Both the tasks resulted in amplitude modulation of an approximately 420–700 ms latency ERP component at parietal electrodes. Scalp topographies indicated that right parietal cortex was activated during mental rotation, but bilateral parietal regions were activated during mental paper folding. Our results support the notion of a right hemispheric superiority for tasks involving simple, single mental rotations, but indicate greater involvement of the left hemisphere when a more complex sequence of transformations are required. This task-dependent lability of hemispheric function may account for some of the inconsistent results reported by previous neuroimaging and electrophysiological studies.

Introduction

Mental imagery involves the generation and manipulation of mental representations in the absence of appropriate sensory input. The most extensively studied mental-image manipulation is mental rotation, which is the ability to imagine visual shapes rotated to an orientation other than that in which they appear. Since reaction times are longer for larger angles of misorientation, it has been proposed that mental representations of objects are rotated through a trajectory in much the same manner that physical objects are rotated. Assuming a constant rate of mental rotation, it would thus take longer to traverse larger angular displacements (Cooper & Shepard, 1973; Shepard & Metzler, 1971).

Cooper and Shepard (1973) introduced a simple paradigm for studying mental rotation in which they presented asymmetric alphanumeric characters and required subjects to decide if each was normal or mirror-reversed (parity-judgment). These authors reasoned that if such characters are internally represented in their normal upright (canonical) positions, misoriented images would have to be mentally-rotated in order to perform the discrimination. Since the normal and mirror-reversed characters possess the same features, this task ensured that the discrimination could not be accomplished by alternative feature-based strategies that are independent of orientation. Indeed, the observed RT functions were entirely consistent with a mental rotation strategy, showing a sharp increase as a function of the angular departure of characters from their upright positions. These RT functions have been extensively replicated in parity-judgment tasks involving the discrimination of not only mirror-imaged characters, but also of left and right hands (Cooper & Shepard, 1975) of the mirror-image polygons (Ely, 1982), and of left- and right-facing naturalistic objects (Jolicoeur, 1985). Further, these RT functions are unique to parity judgment tasks: to our knowledge the only comparable phenomenon is the increased RTs obtained when subjects are required to identify rotated characters (Corballis, Zbrodoff, Shetzer, & Butler, 1978; White, 1980) or naturalistic objects (Jolicoeur, 1985). However, these tasks typically produce shallower RT slopes and a “dip” at 180°, and it is debatable whether mental rotation is actually involved (see Jolicoeur, 1990, Murray, 1997) for more detailed treatments of this issue). Thus, there is good evidence that the sharp RT functions elicited by parity judgments of misoriented shapes are a unique “behavioral signature” of mental rotation.

If mental rotations are analogous to physical rotations, then this type of processing must be quite different from other types of computations that occur in the brain. The observation that reaction–time functions for mental rotation are continuous (such that intermediate RTs will be obtained for intermediate angles) indicates that it is implemented with an analogue type of code. If the computations involved were digital (for example), then there is no reason to expect processing time to increase with angular orientation in this fashion. Thus, mental rotation seems to require a code that is fundamentally different from that required by other important mental processes such as language (c.f. Corballis, 1997).

There is considerable interest in how a mental process with analogue properties might be implemented in the brain. Two lines of neurophysiological evidence are particularly pertinent to this issue. First, Georgopoulos (1995), Georgopoulos and Pellizzer (1995), Georgopoulos, Lurito, Petrides, Schwartz, and Massey (1989), and Georgopoulos, Taira, and Lukashin (1993) have demonstrated that an imagined movement is represented by an analogue sweep of the neuronal population vector in primary motor cortex of monkeys. Second, there is substantial convergent evidence from neuropsychological studies (e.g. Farah & Hammond, 1988), functional brain imaging (e.g. Alivisatos & Petrides, 1997; Harris et al., 2000; Kosslyn, Digirolamo, Thompson, & Alpert, 1998; Tagaris et al., 1996, Tagaris et al., 1997), and electrophysiological recordings (Peronnet & Farah, 1989; Stuss, Sarazin, Leech, & Picton, 1983) that mental rotation also draws upon processes in the parietal lobe of the brain. Thus, current evidence supports the feasibility of analogue processing at the neuronal level, and points to the parietal lobe as a particularly important zone for the computations underlying mental rotation.

Mental rotation may also be more dependent on processes in the right than in the left hemisphere. One commissurotomized person, for example, proved initially unable to mentally rotate letters or simple stick figures when they were presented to his right visual field and, thus to the left hemisphere, but was able to do so when they were presented to his left visual field and right hemisphere (Corballis & Sergent, 1989). Although his left hemisphere gained some proficiency in later testing, it remained inferior to the right, and may have adopted strategies other than analogue rotation. In support of this observation, Farah and Hammond (1988) reported that patients exhibit a deficit in mental rotation following right, but not left, parietal damage, and Harris et al. (2000), in a PET study, found selective activation in the right parietal lobe during a mental-rotation task.

However, not all studies have shown the right hemisphere to be critical for mental rotation. Mehta and Newcombe (1991) reported that patients with lesions restricted to the left hemisphere show deficits on mental-rotation tasks. Contrary to the evidence of Harris et al. (2000), other brain-imaging studies have suggested bilateral parietal involvement rather than exclusive right-hemisphere involvement (e.g. Kosslyn et al., 1998, Tagaris et al., 1996, Tagaris et al., 1997). These studies, however, used a mental-rotation task similar to that devised by Shepard and Metzler (1971), in which subjects rotate unfamiliar three-dimensional torus shapes, whereas Harris et al. (2000) used the simpler Cooper and Shepard (1973) task, in which subjects rotate familiar alphanumeric characters in two-dimensional space. There is some evidence that the Shepard–Metzler figures are rotated in piecemeal fashion (Bethell-Fox & Shepard, 1988; Just & Carpenter, 1985), and it may be this aspect, rather than the rotation component itself, that favors the left hemisphere (cf. Corballis, 1991). More generally, the left hemisphere may be increasingly engaged in spatial performance as the complexity of the task increases (De Renzi, 1978; McGuinness & Bartell, 1982).

This suggests that Cooper and Shepard’s task (1973), in which the subjects are presented with asymmetrical normal or mirror-reversed forms of alphanumeric characters presented at varying orientations from upright and are required to determine whether these characters are normal or mirror-reversed, may be a better measure of pure mental rotation, uncontaminated by piecemeal processing. Yet even this task has failed to yield consistent results in neuroimaging studies. Harris et al. (2000) did find right-parietal activation, but Alivisatos and Petrides (1997) found activation in the left inferior and posterio-superior parietal cortices and Tagaris et al. found bilateral activation of parietal areas (Tagaris et al., 1997). Thus, the issue of whether mental rotation processes are lateralised to one hemisphere or the other has not yet been resolved.

A difficulty with studies using PET or fMRI is that these techniques have relatively poor temporal resolution. Typically, PET involves averaging activity over a minute or more while most fMRI paradigms yield a temporal resolution of approximately 6 s. As a consequence these techniques are likely to provide a temporally smeared image of sensory, perceptual, cognitive and motor elements of a given mental-rotation task. In the present study, we attempt to circumvent this problem by using EEG to locate the mental-rotation component in time. The utility of EEG in this context derives from the fact that mental rotation has well-established electrophysiological markers. Peronnet and Farah (1989) measured event-related potentials (ERPs) in response to misoriented alphanumeric characters, and found a negative going modulation of a 400–800 ms latency component at parietal electrodes. This modulation was correlated with angular departure of the stimuli from upright, suggesting that it was related to mental rotation itself. These findings have been extensively replicated in more recent research (e.g. Heil, Rauch, & Hennighausen, 1998). In the present study, we used the mental-rotation task of Cooper and Shepard (1973), because this is uncontaminated by sequential, piecemeal processing, and as such provides a relatively pure measure of the mental rotation. We wished to characterize the spatial distribution of mental rotation ERPs using a high-density, 128-electrode montage, in order to address questions concerning hemispheric lateralization. Previous EEG studies have not employed more than about 50 electrodes.

We also wished to compare brain responses obtained in this relatively pure mental-rotation task to those obtained with a more complex mental transformation. Shepard and Feng (1972) devised a mental paper-folding task in which subjects must imagine re-folding the six interconnected squares representing an unfolded and flattened cube. Each individual fold can be performed by a simple mental rotation of a square (out of the picture plane), but multiple folds require a coordinated sequence of these transformations. As there is evidence that the left hemisphere is specialized for piecewise decomposition of shapes (see Corballis, 1991 for a review), we hypothesized that mental paper folding may invoke greater left hemisphere activity than simple mental rotation. To our knowledge, there have been no previous electrophysiological or neuroimaging studies of the Shepard and Feng (1972) mental paper-folding task.

Section snippets

Subjects

Eighteen neurological normal subjects were recruited from students and faculty at the University of Auckland, New Zealand, and paid NZ$ 20.00 for approximately 2.5 h of participation. All were right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971) (mean rating of 88.57, range from 80 to 100). The procedures were approved by the University of Auckland Human Subjects Ethics Committee. Four subjects were dropped from analysis due to ocular and/or movement artefact during

Mental-rotation task

Fig. 2(A) shows the reaction times as a function of angle of orientation for normal and mirror-reversed letters. Overall, the reaction times for mirror-reversed letters were significantly longer than the reaction times for normal letters (F(1,13)=15.19, P=0.002), by an average of 160 ms. There was also a significant main effect of orientation (F(3,39)=79.08, P<0.001), which was dominated by a linear trend (F(1,13)=106.65, P<0.001), but the quadratic (F(1,13)=27.98, P<0.001) and cubic (F

Discussion

The present experiment compared electrophysiological brain responses measured, with a high density of spatial sampling, during two tasks that required subjects to perform mental transformations of visual images: a relatively simple task whose correct performance involved a single, holistic mental transformation (mental rotation of a letter in the picture plane), and a more complicated task whose correct performance clearly demanded a coordinated sequence of mental transformations (paper folds,

Acknowledgements

This work was supported by the Royal Society of New Zealand Marsden Fund Grant UOA813. We thank the editor and two anonymous reviewers for helpful comments on an earlier version of this article.

References (45)

  • D. Lehmann et al.

    Spatial analysis of evoked potentials in man—a review

    Progress in Neurobiology

    (1984)
  • D. McGuinness et al.

    Lateral asymmetry: Hard or simple-minded?

    Neuropsychologia

    (1982)
  • R.C. Oldfield

    The assessment and analysis of handedness: The Edinburgh Inventory

    Neuropsychologia

    (1971)
  • F. Peronnet et al.

    Mental rotation: An event-related potential study with a validated mental-rotation task

    Brain and Cognition

    (1989)
  • R.N. Shepard et al.

    A chronometric study of mental paper folding

    Cognitive Psychology

    (1972)
  • D.T. Stuss et al.

    Event-related potentials during naming and mental rotation

    Electroencephalography and Clinical Neurophysiology

    (1983)
  • D.M. Tucker

    Spatial sampling of head electrical fields: The geodesic sensor net

    Electroencephalography and Clinical Neurophysiology

    (1993)
  • C. Bethell-Fox et al.

    Mental rotation: Effects of complexity and familiarity

    Journal of Experimental Psychology: Human Perception and Performance

    (1988)
  • Cooper, L. A., & Shepard, R. N. (1973). Chronometric studies of the rotation of mental images. In W. G. Chase (Ed.),...
  • L.A. Cooper et al.

    Mental transformations in the identification of left and right hands

    Journal of Experimental Psychology: Human Perception and Performance

    (1975)
  • Corballis, M. C. (1991). The lopsided ape. New York: Oxford University...
  • M.C. Corballis et al.

    Decisions about identity and orientation of rotated letters and digits

    Memory and Cognition

    (1978)
  • Cited by (57)

    • Attentional resource allocation among individuals with different fluid intelligence: The integrated control hypothesis and its evidence from pupillometry

      2022, Neuropsychologia
      Citation Excerpt :

      In addition, from the interview we also noticed that the low-difficulty trails in the exploration task may show fewer exploratory features because they were too easy, while medium- and high-difficulty trials exactly showed exploratory features. According to previous studies (Milivojevic et al., 2003; Shepard and Feng, 1972), the difficulty level of this task was classified by the total number of squares involved in the match-mismatch decision (see Shepard and Feng, 1972 for more detail). As shown in Fig. 3, the low-, medium-, and high-difficulty problems involved folding two, four and six squares, respectively.

    View all citing articles on Scopus
    View full text