Abstract
In traditional video conferencing systems, it is impossible for users to have eye contact when looking at the conversation partner’s face displayed on the screen, due to the disparity between the locations of the camera and the screen. In this work, we implemented a gaze correction system that can automatically maintain eye contact by replacing the eyes of the user with the direct looking eyes (looking directly into the camera) captured in the initialization stage. Our real-time system has good robustness against different lighting conditions and head poses, and it provides visually convincing and natural results while relying only on a single webcam that can be positioned almost anywhere around the screen.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jones, A., Lang, M., Fyffe, G., Yu, X., Busch, J., McDowall, I., Bolas, M., Debevec, P.: Achieving eye contact in a one-to-many 3d video teleconferencing system. ACM Trans. Graph. (TOG) 28, 64:1–64:8 (2009)
Okada, K.I., Maeda, F., Ichikawaa, Y., Matsushita, Y.: Multiparty videoconferencing at virtual social distance: Majic design. In: Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work, pp. 385–393. ACM (1994)
Kuster, C., Popa, T., Bazin, J.C., Gotsman, C., Gross, M.: Gaze correction for home video conferencing. ACM Trans. Graph. (TOG) 31, 174 (2012)
Zhu, J., Yang, R., Xiang, X.: Eye contact in video conference via fusion of time-of-flight depth sensor and stereo. 3D Res. 2, 1–10 (2011)
Criminisi, A., Shotton, J., Blake, A., Torr, P.H.: Gaze manipulation for one-to-one teleconferencing. In: Ninth IEEE International Conference on Computer Vision, 2003, Proceedings, pp. 191–198. IEEE (2003)
Yang, R., Zhang, Z.: Eye gaze correction with stereovision for video-teleconferencing. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part II. LNCS, vol. 2351, pp. 479–494. Springer, Heidelberg (2002)
Cham, T.J., Krishnamoorthy, S., Jones, M.: Analogous view transfer for gaze correction in video sequences. In: 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002, vol. 3, pp. 1415–1420. IEEE (2002)
Yip, B., Jin, J.S.: An effective eye gaze correction operation for video conference using antirotation formulas. In: Proceedings of the 2003 Joint Conference of the Fourth International Conference on Information, Communications and Signal Processing and Fourth Pacific Rim Conference on Multimedia, vol. 2, pp. 699–703. IEEE (2003)
Gemmell, J., Toyama, K., Zitnick, C.L., Kang, T., Seitz, S.: Gaze awareness for video-conferencing: a software approach. IEEE Multimedia 7, 26–35 (2000)
Gemmell, J., Zhu, D.: Implementing gaze-corrected videoconferencing. In: Communications, Internet, and Information Technology, pp. 382–387 (2002)
Giger, D., Bazin, J.C., Kuster, C., Popa, T., Gross, M.: Gaze correction with a single webcam. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2014)
Hundt, E., Riegel, T., Schwartzel, H., Ziegler, M.: Correction of the gaze direction for a videophone, US Patent 5,499,303 (1996)
Jerald, J., Daily, M.: Eye gaze correction for videoconferencing. In: Proceedings of the 2002 Symposium on Eye Tracking Research & Applications, pp. 77–81. ACM (2002)
Weiner, D., Kiryati, N.: Virtual gaze redirection in face images. In: 12th International Conference on Image Analysis and Processing, 2003, Proceedings, pp. 76–81. IEEE (2003)
Wolf, L., Freund, Z., Avidan, S.: An eye for an eye: a single camera gaze-replacement method. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 817–824. IEEE (2010)
Kononenko, D., Lempitsky, V.: Learning to look up: realtime monocular gaze correction using machine learning. In: Computer Vision and Pattern Recognition (CVPR). IEEE (2015)
Smith, B.A., Yin, Q., Feiner, S.K., Nayar, S.K.: Gaze locking: passive eye contact detection for human-object interaction. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp. 271–280. ACM (2013)
Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 532–539. IEEE (2013)
Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25, 218–233 (2003)
Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. ACM Trans. Graph. (TOG) 27, 39 (2008)
Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. (TOG) 2, 217–236 (1983)
Acknowledgements
This work was supported in part by Citrix Online and by NSF Grants IIS-1423676 and IIS-1219261.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Qin, Y., Lien, KC., Turk, M., Höllerer, T. (2015). Eye Gaze Correction with a Single Webcam Based on Eye-Replacement. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2015. Lecture Notes in Computer Science(), vol 9474. Springer, Cham. https://doi.org/10.1007/978-3-319-27857-5_54
Download citation
DOI: https://doi.org/10.1007/978-3-319-27857-5_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27856-8
Online ISBN: 978-3-319-27857-5
eBook Packages: Computer ScienceComputer Science (R0)