Skip to main content
SpringerLink
Log in

All Smiles are Not Created Equal: Morphology and Timing of Smiles Perceived as Amused, Polite, and Embarrassed/Nervous

  • Original Paper
  • Published:
Journal of Nonverbal Behavior Aims and scope Submit manuscript

Abstract

We investigated the correspondence between perceived meanings of smiles and their morphological and dynamic characteristics. Morphological characteristics included co-activation of Orbicularis oculi (AU 6), smile controls, mouth opening, amplitude, and asymmetry of amplitude. Dynamic characteristics included duration, onset and offset velocity, asymmetry of velocity, and head movements. Smile characteristics were measured using the Facial Action Coding System (Ekman et al. 2002) and Automated Facial Image Analysis (Cohn and Kanade 2007). Observers judged 122 smiles as amused, embarrassed, nervous, polite, or other. Fifty-three smiles met criteria for classification as perceived amused, embarrassed/nervous, or polite. In comparison with perceived polite, perceived amused more often included AU 6, open mouth, smile controls, larger amplitude, larger maximum onset and offset velocity, and longer duration. In comparison with perceived embarrassed/nervous, perceived amused more often included AU 6, lower maximum offset velocity, and smaller forward head pitch. In comparison with perceived polite, perceived embarrassed/nervous more often included mouth opening and smile controls, larger amplitude, and greater forward head pitch. Occurrence of the AU 6 in perceived embarrassed/nervous and polite smiles questions the assumption that AU 6 with a smile is sufficient to communicate felt enjoyment. By comparing three perceptually distinct types of smiles, we found that perceived smile meanings were related to specific variation in smile morphological and dynamic characteristics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. We used a conservative measure of the number of action units as it was based on the presence/absence of AUs. Hence, each AU could have a maximum frequency of 1 per smile regardless of how many times it actually occurred during the smile.

  2. Heterogeneity of variance among the smile types was not uncommon. For this reason, data were analyzed both with and without assumptions for homogeneity. The pattern of findings remained unchanged. For consistency, the results reported are for tests that assume homogeneity of variance.

References

  • Abe, J. A. A., Beetham, M., & Izard, C. E. (2002). What do smiles mean? An analysis in terms of differential emotions theory. In M. H. Abel (Ed.), An empirical reflection on the smile (Vol. 4, pp. 83–109). Lewiston, NY: The Edwin Mellen Press.

    Google Scholar 

  • Bonanno, G. A., Keltner, D., Noll, J. G., Putnam, F. W., Trickett, P. K., LeJeune, J., et al. (2002). When the face reveals what words do not: Facial expressions of emotion, smiling, and the willingness to disclose childhood sexual abuse. Journal of Personality and Social Psychology, 83, 94–110.

    Article  PubMed  Google Scholar 

  • Brannigan, C. R., & Humphries, D. A. (1972). Human non-verbal behaviour, a means of communication. In N. B. Jones (Ed.), Ethological studies of child behaviour (pp. 37–64). Oxford, UK: Cambridge University Press.

    Google Scholar 

  • Brown, W. M., & Moore, C. (2002). Smile asymmetries and reputation as reliable indicators of likelihood to cooperate: An evolutionary analysis. In S. P. Shohov (Ed.), Advances in psychology research (Vol. 11, pp. 59–78). Huntington, NY: Nova Science.

    Google Scholar 

  • Carroll, J. M., & Russell, J. A. (1996). Do facial expressions signal specific emotions? Judging emotion from the face in context. Journal of Personality and Social Psychology, 70, 205–218.

    Article  PubMed  Google Scholar 

  • Cheyne, J. A. (1976). Development of forms and functions of smiling in preschoolers. Child Development, 47, 820–823.

    Article  Google Scholar 

  • Cohn, J. F., & Kanade, T. (2007). Use of automated facial image analysis for measurement of emotion expression. In J. A. Coan & J. B. Allen (Eds.), The handbook of emotion elicitation and assessment (pp. 222–238). New York: Oxford University Press.

    Google Scholar 

  • Cohn, J. F., Reed, L. I., Moriyama, T., Xiao, J., Schmidt, K. L., & Ambadar, Z. (2004). Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles and gesture recognition. Paper Presented at the Sixth IEEE International Conference on Automatic Face and Gesture Recognition.

  • Cohn, J. F., Zlochower, A. J., Lien, J., & Kanade, T. (1999). Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. Psychophysiology, 36, 35–43.

    Article  PubMed  Google Scholar 

  • Costa, M., Dinsbach, W., Manstead, A. S. R., & Ricci Bitti, P. E. (2001). Social presence, embarrassment and nonverbal behavior. Journal of Nonverbal Behavior, 25, 225–240.

    Article  Google Scholar 

  • Dinehart, L. H. B., Messinger, D. S., Acosta, S. I., Cassel, T., Ambadar, Z., & Cohn, J. (2005). Adult perceptions of positive and negative infant emotional expressions. Infancy, 8, 279–303.

    Article  Google Scholar 

  • Ekman, P. (1992). Telling lies. Cues to deceit in the marketplace, politics, and marriage. New York: WW. Norton & Company.

    Google Scholar 

  • Ekman, P. (2007). The directed facial action task: Emotional responses without appraisal. In J. A. Coan & J. B. Allen (Eds.), The handbook of emotion elicitation and assessment (pp. 47–53). New York: Oxford University Press.

    Google Scholar 

  • Ekman, P., Davidson, R. J., & Friesen, W. V. (1990). The Duchenne smile: Emotional expression and brain physiology II. Journal of Personality and Social Psychology, 58, 342–353.

    Article  PubMed  Google Scholar 

  • Ekman, P., & Friesen, W. V. (1982). Felt, false, and miserable smiles. Journal of Nonverbal Behavior, 6, 238–252.

    Article  Google Scholar 

  • Ekman, P., Friesen, W. V., & Hager, J. (2002). Facial action coding system (FACS): A technique for the measurement of facial movement. Salt Lake City, UT: Research Nexus.

    Google Scholar 

  • Ekman, P., Friesen, W. V., & O’Sullivan, M. (1988). Smiles when lying. Journal of Personality and Social Psychology, 54, 414–420.

    Article  PubMed  Google Scholar 

  • Ekman, P., Hager, J. C., & Friesen, W. V. (1981). The symmetry of emotional and deliberate facial actions. Psychophysiology, 18, 101–106.

    Article  PubMed  Google Scholar 

  • Fernández-Dols, J. M., Wallbott, H., & Sanchez, F. (1991). Emotion category accessibility and the decoding of emotion from facial expression and context. Journal of Nonverbal Behavior, 15, 107–123.

    Article  Google Scholar 

  • Frank, M. G., Ekman, P., & Friesen, W. V. (1993). Behavioral markers and recognizability of the smile of enjoyment. Journal of Personality and Social Psychology, 64, 83–93.

    Article  PubMed  Google Scholar 

  • Fridlund, A. J. (1991). Sociality of solitary smiling: Potentiation by an implicit audience. Journal of Personality and Social Psychology, 60, 229–240.

    Article  Google Scholar 

  • Garotti, P. L., Caterina, R., Brighetti, G., Giberti, E., & Ricci-Bitti, P. E. (1993). The role of the smile as an indicator of different emotional states. Rassegna di Psicologia, 10, 29–53.

    Google Scholar 

  • Gosselin, P., Perron, M., Legault, M., & Campanella, P. (2002). Children’s and adults’ knowledge of the distinction between enjoyment and nonenjoyment smiles. Journal of Nonverbal Behavior, 26, 83–108.

    Article  Google Scholar 

  • Haidt, J., & Keltner, D. (1999). Culture and facial expression: Open ended methods find more expressions and a gradient of recognition. Cognition and Emotion, 13, 225–266.

    Article  Google Scholar 

  • Harrigan, J. A., & Taing, K. T. (1997). Fooled by a smile: Detecting anxiety in others. Journal of Nonverbal Behavior, 21, 203–221.

    Article  Google Scholar 

  • Harris, C. R. (2001). Cardiovascular responses of embarrassment and effects of emotional suppression in a social setting. Journal of Personality and Social Psychology, 81, 886–897.

    Article  PubMed  Google Scholar 

  • Harris, C. R., & Alvarado, N. (2005). Facial expressions, smile types, and self-report during humour, tickle, and pain. Cognition and Emotion, 19, 655–669.

    Article  Google Scholar 

  • Hess, U., Adams, R. B., & Kleck, R. E. (2007). Looking at you or looking elsewhere: The influence of head orientation on the signal value of emotional facial expressions. Motivation and Emotion, 31, 137–144.

    Article  Google Scholar 

  • Hess, U., Banse, R., & Kappas, A. (1995). The intensity of facial expression is determined by underlying affective state and social situation. Journal of Personality and Social Psychology, 69, 280–288.

    Article  Google Scholar 

  • Hess, U., Beaupré, M., & Cheung, N. (2002). Who to whom and why––cultural differences and similarities in the function of smiles. In M. H. Abel (Ed.), An empirical reflection on the smile (pp. 187–216). Lewiston, NY: The Edwin Mellen Press.

    Google Scholar 

  • Hess, U., Kappas, A., McHugo, G. J., Kleck, R. E., & Lanzetta, J. T. (1989). An analysis of the encoding and decoding of spontaneous and posed smiles: The use of facial electromyography. Journal of Nonverbal Behavior, 13, 121–137.

    Article  Google Scholar 

  • Hess, U., & Kleck, R. E. (1990). Differentiating emotion elicited and deliberate emotional facial expressions. European Journal of Social Psychology, 20, 369–385.

    Article  Google Scholar 

  • Hess, U., & Kleck, R. E. (1994). The cues decoders use in attempting to differentiate emotion-elicited and posed facial expressions. European Journal of Social Psychology, 24, 367–381.

    Article  Google Scholar 

  • Kanade, T., Cohn, J. F., & Tian, Y. (2000). Comprehensive database for facial expression analysis. Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG’00) (pp. 46–53). Grenoble, France.

  • Kappas, A., Hess, U., Barr, C. L., & Kleck, R. E. (1994). Angle of regard: The effect of vertical viewing angle on the perception of facial expressions. Journal of Nonverbal Behavior, 18, 263–280.

    Article  Google Scholar 

  • Keltner, D. (1995). Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame. Journal of Personality and Social Psychology, 68, 441–454.

    Article  Google Scholar 

  • Kraut, R. E., & Johnston, R. E. (1979). Social and emotional messages of smiling: An ethological approach. Journal of Personality and Social Psychology, 37, 1539–1553.

    Article  Google Scholar 

  • Krull, D. S., & Dill, J. C. (1998). Do smiles elicit more inferences than do frowns? The effect of emotional valence on the production of spontaneous inferences. Personality and Social Psychology Bulletin, 24, 289–300.

    Article  Google Scholar 

  • Krumhuber, E., & Kappas, A. (2005). Moving smiles: The role of dynamic components for the perception of the genuineness of smiles. Journal of Nonverbal Behavior, 29, 3–24.

    Article  Google Scholar 

  • Krumhuber, E., Manstead, A. R., & Kappas, A. (2007). Temporal aspects of facial displays in person and expression perception: The effects of smile dynamics, head-tilt, and gender. Journal of Nonverbal Behavior, 29, 3–24.

    Article  Google Scholar 

  • Liu, Y., Schmidt, K. L., Cohn, J. F., & Mitra, S. (2003). Facial asymmetry quantification for expression invariant human identification. Computer Vision and Image Understanding, 91, 138–159.

    Article  Google Scholar 

  • Lyons, M. J., Campbell, R., Plante, A., Coleman, M., Kamachi, M., & Akamatsu, S. (2000). Viewpoint dependent facial expression recognition. Japanese Noh mask and human face. Paper Presented at the 22nd Annual Conference of the Cognitive Science Society, Philadelphia.

  • McGee, A. M., & Skinner, M. (1987). Facial asymmetry and the attribution of personality traits. British Journal of Social Psychology, 26, 181–184.

    PubMed  Google Scholar 

  • Merkel, K., Schmidt, K., Levenstein, R., VanSwearingen, J., & Bentley, B. (2007). Positive affect predicts improved lip movement in facial movement disorder. Otolaryngology - Head and Neck Surgery, 137, 100–104.

    Article  PubMed  Google Scholar 

  • Messinger, D. S. (2002). Positive and negative: Infant facial expressions and emotions. Current Directions in Psychological Science, 11, 1–6.

    Article  Google Scholar 

  • Messinger, D. S., Fogel, A., & Dickson, K. L. (1999). What’s in a smile? Developmental Psychology, 35, 701–708.

    Article  PubMed  Google Scholar 

  • Messinger, D. S., Fogel, A., & Dickson, K. L. (2001). All smiles are positive, but some smiles are more positive than others. Developmental Psychology, 37, 642–653.

    Article  PubMed  Google Scholar 

  • Mignault, A., & Chaudhuri, A. (2003). The many faces of a neutral face: Head tilt and perception of dominance and emotion. Journal of Nonverbal Behavior, 27, 111–132.

    Article  Google Scholar 

  • Miles, L., & Johnston, L. (2007). Detecting happiness: Perceiver sensitivity to enjoyment and non-enjoyment smiles. Journal of Nonverbal Behavior, 31, 259–275.

    Article  Google Scholar 

  • Moriyama, T., Xiao, J., Cohn, J. F., & Kanade, T. (2006). Meticulously detailed eye region model and its application to analysis of facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 738–752.

    Article  PubMed  Google Scholar 

  • Motley, M., & Camden, C. T. (1988). Facial expression of emotion: A comparison of posed expressions versus spontaneous expressions in an interpersonal communication setting. Western Journal of Speech Communication, 52, 1–22.

    Google Scholar 

  • Nijholt, A. (2002). Embodied agents: A new impetus to humor research. Paper Presented at the April Fools’ Day Workshop on Computational Humour, Trento, Italy.

  • Otta, E., Folladore Abrosio, F., & Hoshino, R. L. (1996). Reading a smiling face: Messages conveyed by various forms of smiling. Perceptual and Motor Skills, 82, 1111–1121.

    PubMed  Google Scholar 

  • Pollak, S. D., Cicchetti, D., Hornung, K., & Reed, A. (2000). Recognizing emotion in faces: Developmental effects of child abuse and neglect. Developmental Psychology, 36, 679–688.

    Article  PubMed  Google Scholar 

  • Prkachin, K. M. (1992). The consistency of facial expressions of pain: A comparison across modalities. Pain, 51, 297–306.

    Article  PubMed  Google Scholar 

  • Russell, J. A. (1993). Forced-choice response format in the study of facial expression. Motivation and Emotion, 17, 41–51.

    Article  Google Scholar 

  • Scharlemann, J. P. W., Eckel, C. C., Kacelnik, A., & Wilson, R. K. (2001). The value of a smile: Game theory with a human face. Journal of Economic Psychology, 22, 617–640.

    Article  Google Scholar 

  • Scheff, T. J. (1983). Toward integration in the social psychology of emotions. Annual Reviews of Sociology, 9, 333–354.

    Article  Google Scholar 

  • Schmidt, K. L., Ambadar, Z., & Cohn, J. F. (2005). Timing of lip corner movement affects perceived genuineness of smiles. University of Pittsburgh.

  • Schmidt, K. L., Ambadar, Z., Cohn, J. F., & Reed, L. I. (2006a). Movement differences between deliberate and spontaneous facial expressions: Zygomaticus major action in smiling. Journal of Nonverbal Behavior, 30, 37–52.

    Article  Google Scholar 

  • Schmidt, K. L., & Cohn, J. F. (2001). Human facial expressions as adaptations: Evolutionary questions in facial expression research. American Journal of Physical Anthropology, 116, 3–24.

    Article  Google Scholar 

  • Schmidt, K. L., Liu, Y., & Cohn, J. F. (2006b). The role of structural facial asymmetry in asymmetry of peak facial expressions. Laterality, 11, 40–51.

    Google Scholar 

  • Skinner, M., & Mullen, B. (1991). Facial asymmetry in emotional expression: A meta-analysis of research. British Journal of Social Psychology, 30, 113–124.

    Google Scholar 

  • Van Swearingen, J. M., Cohn, J. F., & Bajaj-Luthra, A. (1999). Specific impairment of smiling increases severity of depressive symptoms in patients with facial neuromuscular disorders. Journal of Aesthetic Plastic Surgery, 23, 416–423.

    Article  Google Scholar 

  • Vrugt, A., Duijnhouwer, H., & Stam, O. (2004). The interpretation of smiling in different situations. Nederlands Tijdschrift voor de Psychologie en haar Grensgebieden, 59, 55–64.

    Google Scholar 

  • Wallbott, H. G. (1991). The robustness of communication of emotion via facial expression: Emotion recognition from photographs with deteriorated pictorial quality. European Journal of Social Psychology, 21, 89–98.

    Article  Google Scholar 

  • Williams, L. M., Senior, C., David, A. S., Loughland, C. M., & Gordon, E. (2001). In search of the “Duchenne Smile”: Evidence from eye movements. Journal of Psychophysiology, 15, 122–127.

    Article  Google Scholar 

Download references

Acknowledgement

Portions of this work were supported by NIMH grant MH51435.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Pittsburgh, 4323 Sennott Square, 210 South Bouquet Street, Pittsburgh, PA, 15260, USA

    Zara Ambadar, Jeffrey F. Cohn & Lawrence Ian Reed

Authors
  1. Zara Ambadar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey F. Cohn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lawrence Ian Reed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zara Ambadar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ambadar, Z., Cohn, J.F. & Reed, L.I. All Smiles are Not Created Equal: Morphology and Timing of Smiles Perceived as Amused, Polite, and Embarrassed/Nervous. J Nonverbal Behav 33, 17–34 (2009). https://doi.org/10.1007/s10919-008-0059-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10919-008-0059-5

Keywords

Search

Navigation