Skip to main content
SpringerLink
Log in

Photorealistic Facial Wrinkles Removal

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 Workshops (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13848))

Included in the following conference series:

Abstract

Editing and retouching facial attributes is a complex task that usually requires human artists to obtain photo-realistic results. Its applications are numerous and can be found in several contexts such as cosmetics or digital media retouching, to name a few. Recently, advancements in conditional generative modeling have shown astonishing results at modifying facial attributes in a realistic manner. However, current methods are still prone to artifacts, and focus on modifying global attributes like age and gender, or local mid-sized attributes like glasses or moustaches. In this work, we revisit a two-stage approach for retouching facial wrinkles and obtain results with unprecedented realism. First, a state of the art wrinkle segmentation network is used to detect the wrinkles within the facial region. Then, an inpainting module is used to remove the detected wrinkles, filling them in with a texture that is statistically consistent with the surrounding skin. To achieve this, we introduce a novel loss term that reuses the wrinkle segmentation network to penalize those regions that still contain wrinkles after the inpainting. We evaluate our method qualitatively and quantitatively, showing state of the art results for the task of wrinkle removal. Moreover, we introduce the first high-resolution dataset, named FFHQ-Wrinkles, to evaluate wrinkle detection methods.

E. Ramon—This work was done prior to joining Amazon.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adobe Inc.: (Adobe photoshop)

    Google Scholar 

  2. visage-lab: (Visage lab face retouch)

    Google Scholar 

  3. Batool, N., Chellappa, R.: Detection and inpainting of facial wrinkles using texture orientation fields and Markov random field modeling. IEEE Trans. Image Process. 23, 3773–3788 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Alaluf, Y., Patashnik, O., Cohen-Or, D.: Only a matter of style: age transformation using a style-based regression model. ACM Trans. Graph. (TOG) 40, 1–12 (2021)

    Article  Google Scholar 

  5. Song, X., Shao, M., Zuo, W., Li, C.: Face attribute editing based on generative adversarial networks. SIViP 14(6), 1217–1225 (2020). https://doi.org/10.1007/s11760-020-01660-0

    Article  Google Scholar 

  6. Lample, G., Zeghidour, N., Usunier, N., Bordes, A., Denoyer, L., Ranzato, M.: Fader networks: Manipulating images by sliding attributes. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  7. He, Z., Zuo, W., Kan, M., Shan, S., Chen, X.: AttGAN: facial attribute editing by only changing what you want. IEEE Trans. Image Process. 28, 5464–5478 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  8. Ding, C., Kang, W., Zhu, J., Du, S.: InjectionGAN: unified generative adversarial networks for arbitrary image attribute editing. IEEE Access 8, 117726–117735 (2020)

    Article  Google Scholar 

  9. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)

    Google Scholar 

  10. Wu, P.W., Lin, Y.J., Chang, C.H., Chang, E.Y., Liao, S.W.: RelGAN: multi-domain image-to-image translation via relative attributes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5914–5922 (2019)

    Google Scholar 

  11. Goodfellow, I., et al.: Generative adversarial nets. In: Advances In Neural Information Processing Systems, vol. 27 (2014)

    Google Scholar 

  12. Abdal, R., Qin, Y., Wonka, P.: Image2styleGAN++: how to edit the embedded images? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8296–8305 (2020)

    Google Scholar 

  13. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

    Google Scholar 

  14. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

  15. Liu, M., et al.: STGAN: a unified selective transfer network for arbitrary image attribute editing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3673–3682 (2019)

    Google Scholar 

  16. Shafaei, A., Little, J.J., Schmidt, M.: AutoreTouch: automatic professional face retouching. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 990–998 (2021)

    Google Scholar 

  17. Suvorov, R., et al.: Resolution-robust large mask inpainting with Fourier convolutions. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2149–2159 (2022)

    Google Scholar 

  18. Chi, L., Jiang, B., Mu, Y.: Fast Fourier convolution. Adv. Neural. Inf. Process. Syst. 33, 4479–4488 (2020)

    Google Scholar 

  19. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 39, 1856–1867 (2019)

    Article  Google Scholar 

  20. Yap, M.H., Batool, N., Ng, C.C., Rogers, M., Walker, K.: A survey on facial wrinkles detection and inpainting: datasets, methods, and challenges. IEEE Trans. Emerg. Top. Comput. Intell. 5, 505–519 (2021)

    Article  Google Scholar 

  21. Bastanfard, A., Bastanfard, O., Takahashi, H., Nakajima, M.: Toward anthropometrics simulation of face rejuvenation and skin cosmetic. Comput. Anim. Virtual Worlds 15, 347–352 (2004)

    Article  Google Scholar 

  22. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019)

    Google Scholar 

  23. Park, T., Efros, A.A., Zhang, R., Zhu, J.-Y.: Contrastive learning for unpaired image-to-image translation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12354, pp. 319–345. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_19

    Chapter  Google Scholar 

  24. Nazeri, K., Ng, E., Joseph, T., Qureshi, F., Ebrahimi, M.: EdgeConnect: structure guided image inpainting using edge prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019)

    Google Scholar 

  25. Liao, L., Xiao, J., Wang, Z., Lin, C.-W., Satoh, S.: Guidance and evaluation: semantic-aware image inpainting for mixed scenes. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12372, pp. 683–700. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58583-9_41

    Chapter  Google Scholar 

  26. Yang, J., Qi, Z., Shi, Y.: Learning to incorporate structure knowledge for image inpainting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12605–12612 (2020)

    Google Scholar 

  27. Liao, L., Xiao, J., Wang, Z., Lin, C.W., Satoh, S.: Image inpainting guided by coherence priors of semantics and textures. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6539–6548 (2021)

    Google Scholar 

  28. Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8798–8807 (2018)

    Google Scholar 

  29. Suthar, R., Patel, M.K.R.: A survey on various image inpainting techniques to restore image. Int. J. Eng. Res. Appl. 4, 85–88 (2014)

    Google Scholar 

  30. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  31. Jiang, L., Dai, B., Wu, W., Loy, C.C.: Focal frequency loss for image reconstruction and synthesis. In: ICCV (2021)

    Google Scholar 

  32. Lu, Z., Jiang, J., Huang, J., Wu, G., Liu, X.: Glama: joint spatial and frequency loss for general image inpainting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1301–1310 (2022)

    Google Scholar 

  33. Ross, A., Doshi-Velez, F.: Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  34. Or-El, R., Sengupta, S., Fried, O., Shechtman, E., Kemelmacher-Shlizerman, I.: Lifespan age transformation synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12351, pp. 739–755. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58539-6_44

    Chapter  Google Scholar 

  35. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)

    Google Scholar 

  36. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  37. Yakubovskiy, P.: Segmentation models PyTorch (2020). https://github.com/qubvel/segmentation_models.pytorch

  38. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)

    Google Scholar 

  39. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  40. Buslaev, A., Parinov, A., Khvedchenya, E., Iglovikov, V., Kalinin, A.: Albumentations: fast and flexible image augmentations. ArXiv e-prints (2018)

    Google Scholar 

  41. Newson, A., Almansa, A., Gousseau, Y., Pérez, P.: Non-local patch-based image inpainting. Image Process. Line 7, 373–385 (2017). https://doi.org/10.5201/ipol.2017.189

    Article  MathSciNet  Google Scholar 

  42. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 341–346 (2001)

    Google Scholar 

  43. Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)

  44. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Crisalix S.A, Lausanne, Switzerland

    Marcelo Sanchez, Gil Triginer & Eduard Ramon

  2. Universitat Pompeu Fabra, Barcelona, Spain

    Marcelo Sanchez & Coloma Ballester

  3. Université Gustave Eiffel, Paris, France

    Lara Raad

Authors
  1. Marcelo Sanchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gil Triginer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Coloma Ballester
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lara Raad
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eduard Ramon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcelo Sanchez .

Editor information

Editors and Affiliations

  1. University of Tokyo, Tokyo, Japan

    Yinqiang Zheng

  2. Hacettepe University, Ankara, Türkiye

    Hacer Yalim Keleş

  3. Data61/CSIRO, Canberra, ACT, Australia

    Piotr Koniusz

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sanchez, M., Triginer, G., Ballester, C., Raad, L., Ramon, E. (2023). Photorealistic Facial Wrinkles Removal. In: Zheng, Y., Keleş, H.Y., Koniusz, P. (eds) Computer Vision – ACCV 2022 Workshops. ACCV 2022. Lecture Notes in Computer Science, vol 13848. Springer, Cham. https://doi.org/10.1007/978-3-031-27066-6_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27066-6_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27065-9

  • Online ISBN: 978-3-031-27066-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation