Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Understanding the recognition of facial identity and facial expression

Key Points

  • The dominant view in current theories of face perception is that facial identity (recognizing who a person is) and facial expression (interpreting their moods and feelings) are processed by distinct parallel visual routes. Although there is considerable evidence to support the independent coding of identity and expression, it is not clear whether the idea of distinct parallel visual routes provides the best fit to the data.

  • We conclude that there is clear evidence for some separation between the coding of facial identity and expression, the concept of independent visual pathways is not strongly supported. The data are consistent with other potential frameworks that deserve to be more fully explored.

  • One alternative framework derives from image-based analysis of faces using principal component analysis (PCA). This shows that the independent perception of facial identity and facial expression can be modelled within a single representational framework in which some dimensions (principal components) code facial identity, some code facial expressions and others code both. PCA therefore indicates that the dissociation of identity and expression might be partial rather than absolute.

  • We also focus on Haxby and colleagues' observations that facial expressions and other 'changeable' facial cues (such as lipspeech and gaze) are associated with the inferior occipital gyrus and superior temporal sulcus (STS), whereas 'invariant' facial cues (such as facial identity) are associated with the inferior occipital gyrus and lateral fusiform gyrus. This distinction begs more fundamental questions, such as why are facial characteristics divided in this manner and why is the STS more interested in facial expressions, lipspeech and gaze?

  • One potential explanation lies in the fact that the STS is not only sensitive to changeable facial characteristics, but also to other perceptual dimensions that are inherently linked with them (such as their associated vocalizations and dynamic information). There is evidence that the STS might be involved in the integration of these different channels. Consequently, we propose that the prominent role of the STS in coding changeable facial characteristics might reflect an increased reliance on integrative mechanisms for interpreting changeable social signals.

  • In summary, an approach to face perception that emphasizes the different physical properties and information-processing demands (such as reliance on integrative mechanisms) of different facial characteristics has considerable value. This differs from the classic approach, which has tended to emphasize distinctions based mainly on informational content (for example, identity versus expression).

Abstract

Faces convey a wealth of social signals. A dominant view in face-perception research has been that the recognition of facial identity and facial expression involves separable visual pathways at the functional and neural levels, and data from experimental, neuropsychological, functional imaging and cell-recording studies are commonly interpreted within this framework. However, the existing evidence supports this model less strongly than is often assumed. Alongside this two-pathway framework, other possible models of facial identity and expression recognition, including one that has emerged from principal component analysis techniques, should be considered.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

References

  1. Bruce, V. & Young, A. W. Understanding face recognition. Br. J. Psychol. 77, 305–327 (1986). A theoretical paper that proposes a multiroute model of face perception in which different facial attributes are processed by distinct functional routes.

    PubMed  Google Scholar 

  2. Haxby, J. V., Hoffman, E. A. & Gobbini, M. I. The distributed human neural system for face perception. Trends Cogn. Sci. 4, 223–233 (2000). Presents a neural model of face perception that comprises 'core' and 'extended' systems. The core system differentiates mechanisms for coding changeable facial properties and mechanisms coding invariant facial properties. The extended system includes neural regions that are involved in semantics, language, emotion and attention, which support the recognition of different facial characteristics.

    CAS  PubMed  Google Scholar 

  3. Bruce, V. Influences of familiarity on the processing of faces. Perception 15, 387–397 (1986).

    CAS  PubMed  Google Scholar 

  4. Campbell, R., Brooks, B., de Haan, E. & Roberts, T. Dissociating face processing skills: decisions about lip-read speech, expression and identity. Q. J. Exp. Psychol. A 49A, 295–314 (1996).

    Google Scholar 

  5. Young, A. W., McWeeny, K. H., Hay, D. C. & Ellis, A. W. Matching familiar and unfamiliar faces on identity and expression. Psychol. Res. 48, 63–68 (1986).

    CAS  PubMed  Google Scholar 

  6. Calder, A. J., Young, A. W., Keane, J. & Dean, M. Configural information in facial expression perception. J. Exp. Psychol. Hum. Percept. Perform. 26, 527–551 (2000).

    CAS  PubMed  Google Scholar 

  7. Etcoff, N. L. Selective attention to facial identity and facial emotion. Neuropsychologia 22, 281–295 (1984).

    CAS  PubMed  Google Scholar 

  8. Young, A. W., Newcombe, F., de Haan, E. H. F., Small, M. & Hay, D. C. Face perception after brain injury: selective impairments affecting identity and expression. Brain 116, 941–959 (1993).

    PubMed  Google Scholar 

  9. Bruyer, R. et al. A case of prosopagnosia with some preserved covert remembrance of familiar faces. Brain Cogn. 2, 257–284 (1983).

    CAS  PubMed  Google Scholar 

  10. Tranel, D., Damasio, A. R. & Damasio, H. Intact recognition of facial expression, gender, and age in patients with impaired recognition of face identity. Neurology 38, 690–696 (1988).

    CAS  PubMed  Google Scholar 

  11. Hornak, J., Rolls, E. T. & Wade, D. Face and voice expression identification in patients with emotion and behavioural changes following ventral frontal lobe damage. Neuropsychologia 34, 247–261 (1996).

    CAS  PubMed  Google Scholar 

  12. Hasselmo, M. E., Rolls, E. T. & Baylis, G. C. The role of expression and identity in face-selective responses of neurons in the temporal visual cortex of the monkey. Behav. Brain Res. 32, 203–218 (1989).

    CAS  PubMed  Google Scholar 

  13. George, M. S. et al. Brain-regions involved in recognizing facial emotion or identity: an oxygen-15 PET study. J. Neuropsychiatry Clin. Neurosci. 5, 384–394 (1993).

    CAS  PubMed  Google Scholar 

  14. Sergent, J., Ohta, S., MacDonald, B. & Zuck, E. Segregated processing of identity and emotion in the human brain: a PET study. Vis. Cogn. 1, 349–369 (1994).

    Google Scholar 

  15. Winston, J. S., Henson, R. N. A., Fine-Goulden, M. R. & Dolan, R. J. fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. J. Neurophysiol. 92, 1830–1839 (2004).

    CAS  PubMed  Google Scholar 

  16. Parry, F. M., Young, A. W., Saul, J. S. M. & Moss, A. Dissociable face processing impairments after brain injury. J. Clin. Exp. Neuropsychol. 13, 545–558 (1990).

    Google Scholar 

  17. Shuttleworth, E. C. J., Syring, V. & Allen, N. Further observations on the nature of prosopagnosia. Brain Cogn. 1, 307–322 (1982).

    PubMed  Google Scholar 

  18. Hanley, J. R., Pearson, N. & Young, A. W. Impaired memory for new visual forms. Brain 113, 1131–1148 (1990). Reports the first detailed case study of a patient who showed impaired learning of faces (prosopamnesia) and of visual forms in general after a brain injury.

    PubMed  Google Scholar 

  19. Tippet, L. J., Milller, L. A. & Farah, M. J. Prosopamnesia: a selective impairment in face learning. Cogn. Neuropsychol. 17, 241–255 (2000).

    Google Scholar 

  20. Hanley, J. R., Young, A. W. & Pearson, N. Defective recognition of familiar people. Cogn. Neuropsychol. 6, 179–210 (1989).

    Google Scholar 

  21. Evans, J. J., Heggs, A. J., Antoun, N. & Hodges, J. R. Progressive prosopagnosia associated with selective right temporal lobe atrophy: a new syndrome? Brain 118, 1–13 (1995).

    PubMed  Google Scholar 

  22. Ellis, A. W., Young, A. W. & Critchley, E. M. R. Loss of memory for people following temporal lobe damage. Brain 112, 1469–1483 (1989). Reports the first detailed case study of a patient who showed impaired access to person-specific semantic information from several modalities.

    PubMed  Google Scholar 

  23. Malone, D. R., Morris, H. H., Kay, M. C. & Levin, H. S. Prosopagnosia: a double dissociation between the recognition of familiar and unfamiliar faces. J. Neurol. Neurosurg. Psychiatry 45, 820–822 (1982).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Ekman, P. & Friesen, W. V. Pictures of Facial Affect (Consulting Psychologists, Palo Alto, California, 1976).

    Google Scholar 

  25. Sprengelmeyer, R. et al. Loss of disgust: perception of faces and emotions in Huntington's disease. Brain 119, 1647–1665 (1996).

    PubMed  Google Scholar 

  26. Calder, A. J., Keane, J., Manes, F., Antoun, N. & Young, A. W. Impaired recognition and experience of disgust following brain injury. Nature Neurosci. 3, 1077–1078 (2000).

    CAS  PubMed  Google Scholar 

  27. Rapcsak, S. Z. et al. Fear recognition deficits after focal brain damage: a cautionary note. Neurology 54, 575–581 (2000).

    CAS  PubMed  Google Scholar 

  28. Bowers, D., Bauer, R. M., Coslett, H. B. & Heilman, K. M. Processing of faces by patients with unilateral hemisphere lesions. 1. Dissociation between judgements of facial affect and facial identity. Brain Cogn. 4, 258–272 (1985).

    CAS  PubMed  Google Scholar 

  29. Shallice, T. From Neuropsychology to Mental Structure (Cambridge Univ. Press, Cambridge, 1988).

    Google Scholar 

  30. Duchaine, B. C., Paerker, H. & Nakayama, K. Normal recognition of emotion in a prosopagnosic patient. Perception 32, 827–839 (2003).

    PubMed  Google Scholar 

  31. Bishop, D. V. M. Cognitive neuropsychology and developmental disorders: uncomfortable bedfellows. Q. J. Exp. Psychol. A 50A, 899–923 (1997).

    Google Scholar 

  32. Karmiloff-Smith, A., Scerif, G. & Ansari, D. Double dissociations in developmental disorders? Theoretically misconceived, empirically dubious. Cortex 39, 161–163 (2003).

    PubMed  Google Scholar 

  33. Turk, M. & Pentland, A. Eigenfaces for recognition. J. Cogn. Neurosci. 3, 71–86 (1991).

    CAS  PubMed  Google Scholar 

  34. Burton, A. M., Bruce, V. & Hancock, P. J. B. From pixels to people: a model of familiar face recognition. Cogn. Sci. 23, 1–31 (1999). Presents a model of facial identity recognition in which the front end is implemented as a PCA, and the subsequent cognitive processes are implemented as an interactive activation and competition architecture.

    Google Scholar 

  35. Hancock, P. J. B., Burton, A. M. & Bruce, V. Face processing: human perception and principal components analysis. Mem. Cognit. 24, 21–40 (1996).

    CAS  PubMed  Google Scholar 

  36. Hancock, P. J. B., Bruce, V. & Burton, A. M. A comparison of two computer-based face identification systems with human perception of faces. Vision Res. 38, 2277–2288 (1998).

    CAS  PubMed  Google Scholar 

  37. O'Toole, A. J. et al. The perception of face gender: the role of stimulus structure in recognition and classification. Mem. Cognit. 26, 146–160 (1998).

    CAS  PubMed  Google Scholar 

  38. O'Toole, A. J., Abdi, H., Deffenbacher, K. A. & Valentin, D. Low-dimensional representation of faces in higher dimensions of the face space. J. Opt. Soc. Am. 10, 405–411 (1993).

    Google Scholar 

  39. O'Toole, A. J., Deffenbacher, K. A., Valentin, D. & Abdi, H. Structural aspects of face recognition and the other race effect. Mem. Cognit. 22, 208–224 (1994).

    CAS  PubMed  Google Scholar 

  40. Padgett, C. & Cottrell, G. in Proceedings of the 2nd Joint Symposium on Neural Computation Vol. 5, 91–101 (La Jolla, Univ. California, San Diego, 1995).

    Google Scholar 

  41. Cottrell, G. W., Branson, K. M. & Calder, A. J. in 24th Annual Meeting of the Cognitive Science Society (Fairfax, Virginia, 2002).

    Google Scholar 

  42. Calder, A. J., Burton, A. M., Miller, P., Young, A. W. & Akamatsu, S. A principal component analysis of facial expressions. Vision Res. 41, 1179–1208 (2001). Provides evidence that a PCA of the Ekman and Friesen facial expressions (reference 24) generates statistical dissociation between PCs that code identity and expression.

    CAS  PubMed  Google Scholar 

  43. Costen, N., Craw, I. & Akamatsu, S. in British Machine Vision Association (Scottish Chapter) Meeting (British Machine Vision Association, Glasgow, 1995).

    Google Scholar 

  44. Deffenbacher, K. A., Vetter, T., Johanson, J. & O'Toole, A. J. Facial aging, attractiveness, and distinctiveness. Perception 27, 1233–1243 (1998).

    CAS  PubMed  Google Scholar 

  45. Schweinberger, S. R. & Soukup, G. R. Asymmetric relationships among perceptions of facial identity, emotion, and facial speech. J. Exp. Psychol. Hum. Percept. Perform. 24, 1748–1765 (1998).

    CAS  PubMed  Google Scholar 

  46. Narumoto, J., Okada, T., Sadato, N., Fukui, K. & Yonekura, Y. Attention to emotion modulates fMRI activity in human right superior temporal sulcus. Cogn. Brain Res. 12, 225–231 (2001).

    CAS  Google Scholar 

  47. Sugase, Y., Yamane, S., Ueno, S. & Kawano, K. Global and fine information coded by single neurons in the temporal visual cortex. Nature 400, 869–873 (1999).

    CAS  PubMed  Google Scholar 

  48. Phillips, M. L. et al. Neural responses to facial and vocal expressions of fear and disgust. Proc. R. Soc. Lond. B 265, 1809–1817 (1998).

    CAS  Google Scholar 

  49. Morris, J. S. et al. A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain 121, 47–57 (1998).

    PubMed  Google Scholar 

  50. Lewis, S. et al. Visual processing of facial affect. Neuroreport 14, 1841–1845 (2003).

    PubMed  Google Scholar 

  51. Streit, M. et al. Neurophysiological correlates of the recognition of facial expressions of emotion as revealed by magnetoencephalography. Cogn. Brain Res. 7, 481–491 (1999).

    CAS  Google Scholar 

  52. Kanwisher, N., McDermott, J. & Chun, M. M. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J. Neurosci. 17, 4302–4311 (1997).

    CAS  PubMed  Google Scholar 

  53. McCarthy, G., Puce, A., Core, J. C. & Allison, T. Face-specific processing in the human fusiform gyrus. J. Cogn. Neurosci. 9, 605–610 (1997).

    CAS  PubMed  Google Scholar 

  54. Henson, R., Shallice, T. & Dolan, R. Neuroimaging evidence for dissociable forms of repetition priming. Science 287, 1269–1272 (2000).

    CAS  PubMed  Google Scholar 

  55. Perrett, D. I. et al. Neurones responsive to faces in the temporal cortex: studies of functional organization, sensitivity to identity and relation to perception. Hum. Neurobiol. 3, 197–208 (1984).

    CAS  PubMed  Google Scholar 

  56. Perrett, D. I., Hietanen, J. K., Oram, M. W. & Benson, P. J. Organization and functions of cells responsive to faces in the temporal cortex. Phil. Trans. R. Soc. Lond. B B335, 23–30 (1992).

    Google Scholar 

  57. Young, M. P. & Yamane, S. Sparse population coding of faces in the inferotemporal cortex. Science 256, 1327–1331 (1992). Shows that face-responsive cells in the inferotemporal cortex of monkeys code visual properties, whereas face-responsive cells in the STS code nonvisual properties, such as familiarity or social status.

    CAS  PubMed  Google Scholar 

  58. Yamane, S., Kaji, S. & Kawano, K. What facial features activate face neurones in the inferotemporal cortex in the monkey? Exp. Brain Res. 73, 209–214 (1988).

    CAS  PubMed  Google Scholar 

  59. Rolls, E. T. & Tovee, M. J. Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. J. Neurophysiol. 73, 713–726 (1995).

    CAS  PubMed  Google Scholar 

  60. Seltzer, B. & Pandya, D. N. Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Res. 149, 1–24 (1978).

    CAS  PubMed  Google Scholar 

  61. Bruce, C., Desimone, R. & Gross, C. G. Visual properties of neurons in a polysensory area in superior temporal cortex. J. Neurophysiol. 46, 369–384 (1981).

    CAS  PubMed  Google Scholar 

  62. Adolphs, R., Tranel, D., Damasio, H. & Damasio, A. Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372, 669–672 (1994).

    CAS  PubMed  Google Scholar 

  63. Sprengelmeyer, R. et al. Knowing no fear. Proc. R. Soc. Lond. B 266, 2451–2456 (1999).

    CAS  Google Scholar 

  64. Calder, A. J. et al. Facial emotion recognition after bilateral amygdala damage: differentially severe impairment of fear. Cogn. Neuropsychol. 13, 699–745 (1996).

    Google Scholar 

  65. Calder, A. J., Lawrence, A. D. & Young, A. W. The neuropsychology of fear and loathing. Nature Rev. Neurosci. 2, 352–363 (2001).

    CAS  Google Scholar 

  66. Sprengelmeyer, R. et al. Disgust implicated in obsessive-compulsive disorder. Proc. R. Soc. Lond. B 264, 1767–1773 (1997).

    CAS  Google Scholar 

  67. Calder, A. J., Keane, J., Lawrence, A. D. & Manes, F. Impaired recognition of anger following damage to the ventral striatum. Brain 127, 1958–1969 (2004).

    PubMed  Google Scholar 

  68. Lawrence, A. D., Calder, A. J., McGowan, S. W. & Grasby, P. M. Selective disruption of the recognition of facial expressions of anger. Neuroreport 13, 881–884 (2002).

    PubMed  Google Scholar 

  69. Krolak-Salmon, P. et al. A specific response to disgust modulated by attention in human ventral anterior insula. Ann. Neurol. 53, 446–453 (2003).

    PubMed  Google Scholar 

  70. Morris, J. S. et al. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature 383, 812–815 (1996).

    CAS  PubMed  Google Scholar 

  71. Breiter, H. C. et al. Response and habituation of the human amygdala during visual processing of facial expression. Neuron 17, 875–887 (1996).

    CAS  PubMed  Google Scholar 

  72. Phillips, M. L. et al. A specific neural substrate for perceiving facial expressions of disgust. Nature 389, 495–498 (1997).

    CAS  PubMed  Google Scholar 

  73. Lawrence, A. D. & Calder, A. J. in Emotion, Evolution, and Rationality (eds Evans, D. & Cruse, P.) 15–47 (Oxford Univ. Press, Oxford, 2003).

    Google Scholar 

  74. Scott, S. K. et al. Impaired auditory recognition of fear and anger following bilateral amygdala lesions. Nature 385, 254–257 (1997).

    CAS  PubMed  Google Scholar 

  75. Adolphs, R. & Tranel, D. Intact recognition of emotional prosody following amygdala damage. Neuropsychologia 37, 1285–1292 (1999).

    CAS  PubMed  Google Scholar 

  76. Anderson, A. K. & Phelps, E. A. Intact recognition of vocal expressions of fear following bilateral lesions of the human amygdala. Neuroreport 9, 3607–3613 (1998).

    CAS  PubMed  Google Scholar 

  77. Wicker, B. et al. Both of us disgusted in my insula: the common neural basis of seeing and feeling disgust. Neuron 40, 655–664 (2003).

    CAS  PubMed  Google Scholar 

  78. Gallese, V., Keysers, C. & Rizzolatti, G. A unifying view of the basis of social cognition. Trends Cogn. Sci. 8, 396–403 (2004).

    PubMed  Google Scholar 

  79. Goldman, A. I. & Sekhar Sripada, C. Simulationist models of face-based emotion recognition. Cognition 94, 193–213 (2005).

    PubMed  Google Scholar 

  80. Keane, J., Calder, A. J., Hodges, J. R. & Young, A. W. Face and emotion processing in frontal variant frontotemporal dementia. Neuropsychologia 40, 655–665 (2002).

    PubMed  Google Scholar 

  81. Adolphs, R., Tranel, D. & Damasio, H. Neural systems for recognizing emotion from prosody. Emotion 2, 23–51 (2002).

    PubMed  Google Scholar 

  82. Adolphs, R., Damasio, H., Tranel, D., Cooper, G. & Damasio, A. R. A role for somatosensory cortices in the visual recognition of emotion as revealed by three-dimensional lesion mapping. J. Neurosci. 20, 2683–2690 (2000).

    CAS  PubMed  Google Scholar 

  83. Anderson, A. K. & Phelps, E. A. Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature 411, 305–309 (2001).

    CAS  PubMed  Google Scholar 

  84. Anderson, A. K., Spencer, D. D., Fulbright, R. K. & Phelps, E. A. Contribution of the anteromedial temporal lobes to the evaluation of facial emotion. Neuropsychology 14, 526–536 (2000).

    CAS  PubMed  Google Scholar 

  85. Labar, K. S., LeDoux, J. E., Spencer, D. D. & Phelps, E. A. Impaired fear conditioning following unilateral temporal lobectomy in humans. J. Neurosci. 15, 6846–6855 (1995).

    CAS  PubMed  Google Scholar 

  86. LaBar, K. S. & Phelps, E. A. Arousal-mediated memory consolidation: role of the medial temporal lobe in humans. Psychol. Sci. 9, 490–493 (1998).

    Google Scholar 

  87. Phelps, E. A. et al. Specifying the contributions of the human amygdala to emotional memory: a case study. Neurocase 4, 527–540 (1998).

    Google Scholar 

  88. Adolphs, R., Cahill, L., Schul, R. & Babinsky, R. Impaired declarative memory for emotional material following bilateral amygdala damage in humans. Learn. Mem. 4, 291–300 (1997).

    CAS  PubMed  Google Scholar 

  89. Bechara, A. et al. Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans. Science 269, 1115–1118 (1995).

    CAS  PubMed  Google Scholar 

  90. Barraclough, N. E., Xiao, D., Baker, C. I., Oram, M. W. & Perrett, D. I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 17, 377–391 (2005).

    PubMed  Google Scholar 

  91. Dolan, R. J., Morris, J. S. & de Gelder, B. Crossmodal binding of fear in voice and face. Proc. Natl Acad. Sci. USA 98, 10006–10010 (2001).

    CAS  PubMed  Google Scholar 

  92. Calvert, G. A., Campbell, R. & Brammer, M. J. Evidence from functional magnetic resonance imaging of crossmodal binding in human heteromodal cortex. Curr. Biol. 10, 649–657 (2000). Provides evidence from fMRI that the STS is involved in the integration of seen and heard speech.

    CAS  PubMed  Google Scholar 

  93. Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H. & Martin, A. Unravelling multisensory integration: patchy organization within human STS multisensory cortex. Nature Neurosci. 7, 1190–1192 (2004).

    CAS  PubMed  Google Scholar 

  94. Campbell, R. et al. Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). Cogn. Brain Res. 12, 233–243 (2001).

    CAS  Google Scholar 

  95. Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P. & Pike, B. Voice-selective areas in human auditory cortex. Nature 403, 309–312 (2000).

    CAS  PubMed  Google Scholar 

  96. Desimone, R. & Gross, C. G. Visual areas in the temporal cortex of the macaque. Brain Res. 178, 363–380 (1979).

    CAS  PubMed  Google Scholar 

  97. Oram, M. W. & Perrett, D. I. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J. Neurophysiol. 76, 109–129 (1996). Reports evidence from cell recording in monkeys that the STS is involved in the integration of visual form and motion components of biological motion.

    CAS  PubMed  Google Scholar 

  98. McGurk, H. & MacDonald, J. Hearing lips and seeing voices. Nature 264, 746–748 (1976).

    CAS  PubMed  Google Scholar 

  99. de Gelder, B. & Vroomen, J. The perception of emotions by ear and by eye. Cogn. Emotion 14, 289–311 (2000).

    Google Scholar 

  100. Rosenblum, L. D., Johnson, J. A. & Saldana, H. M. Point-light facial displays enhance comprehension of speech in noise. J. Speech Hear. Res. 39, 1159–1170 (1996).

    CAS  PubMed  Google Scholar 

  101. Tartter, V. C. & Knowlton, K. C. Perception of sign language from an array of 27 moving spots. Nature 289, 676–678 (1981).

    CAS  PubMed  Google Scholar 

  102. Puce, A., Allison, T., Bentin, S., Gore, J. C. & McCarthy, G. Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci. 18, 2188–2199 (1998).

    CAS  PubMed  Google Scholar 

  103. Hoffman, E. A. & Haxby, J. V. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nature Neurosci. 3, 80–84 (2000).

    CAS  PubMed  Google Scholar 

  104. Langton, S. R. H. The mutual influence of gaze and head orientation in the analysis of social attention direction. Q. J. Exp. Psychol. A 53, 825–845 (2000).

    CAS  PubMed  Google Scholar 

  105. Langton, S. R. H., Honeyman, H. & Tessler, E. The influence of head contour and nose angle on the perception of eye-gaze direction. Percept. Psychophys. 66, 752–771 (2004).

    PubMed  Google Scholar 

  106. Langton, S. R. H. & Bruce, V. You must see the point: automatic processing of cues to the direction of social attention. J. Exp. Psychol. Hum. Percept. Perform. 26, 747–757 (2000).

    CAS  PubMed  Google Scholar 

  107. Adams, R. B. & Kleck, R. E. Perceived gaze direction and the processing of facial displays of emotion. Psychol. Sci. 14, 644–647 (2003). Presents evidence that gaze direction affects the perception of emotion from faces.

    PubMed  Google Scholar 

  108. Hanley, J. R., Smith, S. T. & Hadfield, J. I recognise you but I can't place you: an investigation of familiar-only experiences during tests of voice and face recognition. Q. J. Exp. Psychol. A 51A, 179–195 (1998).

    Google Scholar 

  109. Bruce, V. & Valentine, T. in Practical Aspects of Memory: Current Research and Issues Vol. 1 (eds Gruneberg, M. M., Morris, P. E. & Sykes, R. N.) 169–174 (Wiley, Chichester, 1988).

    Google Scholar 

  110. Bonda, E., Petrides, M., Ostry, D. & Evans, A. Specific involvement of human parietal systems and the amygdala in the perception of biological motion. J. Neurosci. 16, 3737–3744 (1996).

    CAS  PubMed  Google Scholar 

  111. Grossman, E. et al. Brain areas involved in perception of biological motion. J. Cogn. Neurosci. 12, 711–720 (2000).

    CAS  PubMed  Google Scholar 

  112. Castelli, F., Happe, F., Frith, U. & Frith, C. Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns. Neuroimage 12, 314–325 (2000).

    CAS  PubMed  Google Scholar 

  113. Allison, T., Puce, A. & McCarthy, G. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278 (2000).

    CAS  PubMed  Google Scholar 

  114. Beauchamp, M. S., Lee, K. E., Haxby, J. V. & Martin, A. fMRI responses to video and point-light displays of moving and manipulable objects. J. Cogn. Neurosci. 15, 991–1001 (2003).

    PubMed  Google Scholar 

  115. Calvert, G. A. et al. Activation of auditory cortex during silent lipreading. Science 276, 593–596 (1997).

    CAS  PubMed  Google Scholar 

  116. Campbell, R., Landis, T. & Regard, M. Face recognition and lipreading: a neurological dissociation. Brain 109, 509–521 (1986).

    PubMed  Google Scholar 

  117. Adams, R. B. & Kleck, R. E. The effects of direct and averted gaze on the perception of facial communicated emotion. Emotion 5, 3–11 (2005).

    PubMed  Google Scholar 

  118. Adams, R. B., Gordon, H. L., Baird, A. A., Ambady, N. & Kleck, R. E. Gaze differentially modulates amygdala sensitivity to anger and fear faces. Science 300, 1536 (2003).

    CAS  PubMed  Google Scholar 

  119. O'Toole, A. J., Roark, D. A. & Abdi, H. Recognizing moving faces: a psychological and neural synthesis. Trends Cogn. Sci. 6, 261–266 (2002).

    PubMed  Google Scholar 

  120. Simons, D. J. & Levin, D. T. Failure to detect changes to people during a real-world interaction. Psychol. Bull. Rev. 5, 644–649 (1998).

    Google Scholar 

  121. Dimberg, U. Facial reactions for facial expressions. J. Psychophys. 19, 643–647 (1982).

    CAS  Google Scholar 

  122. Watkins, K. E., Strafella, A. P. & Paus, T. Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia 41, 989–994 (2003).

    CAS  PubMed  Google Scholar 

  123. Langton, S. R. H. & Bruce, V. Reflexive visual orienting in response to the social attention of others. Vis. Cogn. 6, 541–567 (1999).

    Google Scholar 

  124. Driver, J. et al. Gaze perception triggers reflexive visuospatial orienting. Vis. Cogn. 6, 509–540 (1999).

    Google Scholar 

  125. Friesen, C. K. & Kingstone, A. The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychol. Bull. Rev. 5, 490–495 (1998).

    Google Scholar 

Download references

Acknowledgements

We would like to thank B. Cox for assistance in preparing the figures.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Andrew J. Calder.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Related links

Related links

FURTHER INFORMATION

Calder's homepage

Young's homepage

Glossary

FUNCTIONAL ACCOUNT

A theoretical framework that is based on the information-processing characteristics of a set of cognitive subsystems rather than their underlying neural mechanisms

PROSOPAGNOSIA

A visual agnosia that is largely restricted to face recognition, but leaves intact recognition of personal identity from other identifying cues, such as voices and names. Prosopagnosia is observed after bilateral and, less frequently, unilateral lesions of the inferior occipitotemporal cortex.

ACQUIRED NEUROPSYCHOLOGICAL DISORDERS

Cognitive impairments that follow neurological damage to previously healthy individuals who have no known genetic or developmental disorders.

PRINCIPAL COMPONENT ANALYSIS

(PCA). A statistical technique that been applied to the analysis of faces. Facial images, which are originally described in terms of a large number of variables (for example, the greyscale values of individual pixels), are recoded in relation to a smaller set of basis vectors (principal components) by identifying correlations between sets of pixels across the entire set of facial images.

POPULATION CODING

The idea that objects, such as faces, are coded as distributed patterns of activity across neuronal populations, so that cells show broadly graded responses to a single stimulus, as opposed to an 'all-or-none'-type response pattern.

GRANDMOTHER CELL

A hypothetical cell that responds specifically to a single face (such as one's grandmother). Although no such highly specialized cell has been found so far, the temporal cortex contains cells that respond preferentially to faces or hands, and some are maximally (although not exclusively) responsive to a particular persons' face. These cells are sometimes referred to as 'grandmother cells'.

HEIDER-SIMMEL-LIKE ANIMATIONS

Short animations that depict the movements of geometric shapes in a manner that is normally perceived as suggestive of social interactions among 'human-like' characters. For example, one shape might be perceived as aggressive and intimidating towards another.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Calder, A., Young, A. Understanding the recognition of facial identity and facial expression. Nat Rev Neurosci 6, 641–651 (2005). https://doi.org/10.1038/nrn1724

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn1724

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing