Skip to main content
SpringerLink
Log in

Altering the Conveyed Facial Emotion Through Automatic Reenactment of Video Portraits

  • Conference paper
  • First Online:
Computer Animation and Social Agents (CASA 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1300))

Included in the following conference series:

Abstract

Current facial reenactment techniques are able to generate results with a high level of photo-realism and temporal consistency. Although the technical possibilities are rapidly progressing, recent techniques focus on achieving fast, visually plausible results. Further perceptual effects caused by altering the original facial expressivity of the recorded individual are disregarded. By investigating the influence of altered facial movements on the perception of expressions we aim to generate not only physically possible but truly believable reenactments.

In this paper we perform two experiments using a modified state-of-the-art technique to reenact a video portrait of a person with different expressions gathered from a validated database of motion captured facial expressions to better understand the impact of reenactments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Brainard, D.H.: The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997)

    Article  Google Scholar 

  2. Castillo, S., Legde, K., Cunningham, D.W.: The semantic space for motion-captured facial expressions. Comput. Animation Virtual Worlds 29(3–4), e1823 (2018)

    Article  Google Scholar 

  3. Castillo, S., Wallraven, C., Cunningham, D.W.: The semantic space for facial communication. Comput. Animation Virtual Worlds 25(3–4), 223–231 (2014)

    Article  Google Scholar 

  4. Cunningham, D.W., Kleiner, M., Wallraven, C., Bülthoff, H.H.: Manipulating video sequences to determine the components of conversational facial expressions. ACM Trans. Appl. Percept. 2(3), 251–269 (2005)

    Article  Google Scholar 

  5. Cunningham, D.W., Wallraven, C.: The interaction between motion and form in expression recognition. In: Proceedings of the ACM Symposium on Applied Perception in Graphics and Visualization, pp. 41–44 (2009)

    Google Scholar 

  6. Dale, K., Sunkavalli, K., Johnson, M.K., Vlasic, D., Matusik, W., Pfister, H.: Video face replacement. ACM Trans. Graph. 30(6), 1–10 (2011)

    Article  Google Scholar 

  7. Feng, Y., Wu, F., Shao, X., Wang, Y., Zhou, X.: Joint 3D face reconstruction and dense alignment with position map regression network. In: European Conference on Computer Vision (2018)

    Google Scholar 

  8. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  9. Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., Seitz, S.M.: Being john malkovich. In: European Conference on Computer Vision, pp. 341–353 (2010)

    Google Scholar 

  10. King, D.E.: Dlib-ml: A machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009). http://www.dlib.net

    Google Scholar 

  11. Mori, M., MacDorman, K., Kageki, N.: The uncanny valley. IEEE Robot Autom. Mag. 19(2), 98–100 (2012)

    Article  Google Scholar 

  12. Nusseck, M., Cunningham, D.W., Wallraven, C., Bülthoff, H.H.: The contribution of different facial regions to the recognition of conversational expressions. J. Vis. 8(8), 1–23 (2008)

    Article  Google Scholar 

  13. Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., Graves, A., Kavukcuoglu, K.: Conditional image generation with PixelCNN decoders. In: Advances in Neural Information Processing Systems, pp. 4790–4798 (2016)

    Google Scholar 

  14. Mittal, T., Bhattacharya, U., Chandra, R., et al.: Emotions don’t lie: A deepfake detection method using audio-visual affective cues (2020). arXiv:2003.06711

  15. Thies, J., Zollhöfer, M., Nießner, M.: Deferred neural rendering: Image synthesis using neural textures. ACM Trans. Graph. 38(4), 66:1–66:12 (2019)

    Article  Google Scholar 

  16. Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2face: real-time face capture and reenactment of RGB videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2387–2395 (2016)

    Google Scholar 

  17. Wang, T.C., et al.: Video-to-video synthesis. In: Advances in Neural Information Processing Systems (2018)

    Google Scholar 

  18. Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation. ACM Trans. Graph. 30(4), 1–10 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge funding by the German Science Foundation (DFG MA2555/15-1 “Immersive Digital Reality”).

Author information

Authors and Affiliations

  1. Institut für Computergraphik, TU Braunschweig, Braunschweig, Germany

    Colin Groth, Jan-Philipp Tauscher, Susana Castillo & Marcus Magnor

Authors
  1. Colin Groth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan-Philipp Tauscher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susana Castillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcus Magnor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Colin Groth .

Editor information

Editors and Affiliations

  1. Faculty of Science and Technology, Bournemouth University, Poole, UK

    Feng Tian

  2. Faculty of Media and Communication, Bournemouth University, Poole, UK

    Xiaosong Yang

  3. École Polytechnique Fédérale de Lausa, Lausanne, Switzerland

    Daniel Thalmann

  4. Computer Science Department, Zhejiang University, Hangzhou, China

    Weiwei Xu

  5. Faculty of Media and Communication, Bournemouth University, Poole, UK

    Jian Jun Zhang

  6. MIRALab/C.U.I., University of Geneva, Geneva, Switzerland

    Nadia Magnenat Thalmann

  7. Faculty of Media and Communication, Bournemouth University, Poole, UK

    Jian Chang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Groth, C., Tauscher, JP., Castillo, S., Magnor, M. (2020). Altering the Conveyed Facial Emotion Through Automatic Reenactment of Video Portraits. In: Tian, F., et al. Computer Animation and Social Agents. CASA 2020. Communications in Computer and Information Science, vol 1300. Springer, Cham. https://doi.org/10.1007/978-3-030-63426-1_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63426-1_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63425-4

  • Online ISBN: 978-3-030-63426-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation