Skip to main content
SpringerLink
Log in

Analogies based video editing

  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

A well-produced video always creates a strong impression on the viewer. However, due to the limitations of the camera, the ambient conditions or the skills of the videographer, the quality of captured videos sometimes falls short of one's expectations. On the other hand, we have a vast amount of superbly captured videos available on the web and in digital libraries. In this paper, we propose the novel approach of video analogies that provides a powerful ability to improve the quality of a video by borrowing features from a higher quality video. We want to improve the given target video in order to obtain a higher quality output video. During the matching phase, we find the correspondence between the pair by using feature matching. Then for the target video, we utilize this correspondence to transfer some desired traits of the source video into the target video in order to obtain a new video. Thus, the new video will obtain the desired features from the source video while retaining the merits of the target video. The video analogies technique provides an intuitive mechanism for automatic editing of videos. We demonstrate the utility of the analogies method by considering three applications – colorizing videos, reducing video blurs, and video rhythm adjustment. We describe each application in detail and provide experimental results to establish the efficacy of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graphics Appl. 21(5), 34–41 (2001)

    Google Scholar 

  2. Welsh, T., Ashikhmin, M., Mueller, K.: Transferring color to grayscale images. In: Proceedings of ACM SIGGRAPH'02, vol.~20, pp. 277–280. San Antonio, USA (2002)

  3. Yan, W.-Q., Kankanhalli, M.S.: Colorizing infrared home videos. In: Proceedings of IEEE ICME'03. Baltimore, USA (2003)

  4. Yan, W.-Q., Kankanhalli, M.S.: Erasing video logos based on image inpainting. In: Proceedings of IEEE ICME'02, pp. 521–524. Lausanne, Switzerland (2002)

  5. Mulhem, P., Kankanhalli, M.S., Hasan, H., Ji, Y.: Pivot vector space approach for audio-video mixing. IEEE Multimedia. 28–40 (2003)

  6. Chang, S., Sundaram, H.: Structural and semantic analysis of video. In: Proceedings of IEEE ICME'00, vol.~2, pp. 687–690. New York, USA (2000)

  7. Kender, J., Teo, B.: Video scene segmentation via continuous video coherence. In: Proceedings of IEEE CVPR'98, pp. 367–373. Santa Barbara, USA (1998)

  8. Hertzmann, A., Jacobs, C., Oliver, N., Curless, B., Salesin, D.: Image analogies. In: Proceedings of ACM SIGGRAPH'01, pp. 327–340. Los Angels, USA (2001)

  9. Evans, T.: A program for the solution of geometric analogy intelligence test questions. In: Semantic Information Processing. MIT Press, New York (1968)

    Google Scholar 

  10. Hertzmann, A., Oliver, N., Seitz, S., Curless, B.: Curves analogies. In: Proceedings of Eurographics Workshop on Rendering'02, pp.~233–246. Pisa, Italy (2002)

  11. Gentner, D.: Structure mapping: a theoretical framework for analogy. Cogn. Sci. 7(2), 155–170 (1983)

    Article  Google Scholar 

  12. Winston, P.: Learning and reasoning by analogy. Commun. ACM 23(12) (1980)

  13. Casares, J., Myers, B.A., Long, A.C., Bhatnagar, R., Stevens, S.M., Dabbish, L., Corbett, A.: Simplifying video editing with intelligent interaction. In: Proceedings CHI'02: Human Factors in Computing Systems, pp.~672–673. Minneapolis, MN (2002)

  14. Girgensohn, A., Bly, S., Shipman, F., Boreczky, J., Wilcox, L.: Home video editing made easy-balancing automation and user control. In: Proceedings of INTERACT'01, pp. 464–471. Tokyo, Japan (2001)

  15. Girgensohn, A., Boreczky, J., Chiu, P., Doherty, J., Foote, J., Golovchinsky, G., Uchihashi, S., Wilcox, L.: A semi-automatic approach to home video editing. CHI Lett. 2, 81–89 (2000)

    Google Scholar 

  16. Davis, M.: Editing out video editing. IEEE Multimedia 10(2), 54–64 (2003)

    Article  Google Scholar 

  17. Dorai, C., Venkatesh, S.: Computational media aesthetics: finding meaning beautiful. IEEE Multimedia 8(4), 10–12 (2001)

    Google Scholar 

  18. Dorai, C., Venkatesh, S.: Bridging the semantic gap with computational media aesthetics. IEEE Multimedia 10(2), 15–17 (2003)

    Article  Google Scholar 

  19. Tekalp, M.: Digital Video Processing. Prentice-Hall, New York (1995)

    Google Scholar 

  20. Sand, P., Teller, S.: Video matching. ACM Trans. Graphics (2004)

  21. Chiang, M., Boult, T.: Local blur estimation and super-resolution. In: Proceedings of IEEE CVPR'97, pp. 821–826. San Juan, Puerto Rico (1997)

  22. Ma, W.-Y., Zhang, H.-J.: Blur determination in the compressed domain using DCT information. In: Proceedings of IEEE ICIP'99, vol.~2, pp. 386–390. Kobe, Japan (1999)

  23. Gonzalez, R., Woods, R.: Digital Image Processing. Addison-Wesley, Reading (1992–1993)

    Google Scholar 

  24. Sun, X.-D., Divakaran, A., Manjunath, B.S.: A motion activity descriptor and its extraction in compressed domain. In: Proceedings of IEEE PCM'01, pp. 450–457. Beijing, China (2001)

  25. Pattiand, A., Sezan, M., Tekalp, A.: Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. Image Process. 6(8), 1067–1076 (1997)

    Google Scholar 

  26. Nguyen, N., Milanfar, P.: A computationally efficient super resolution image reconstruction algorithm. IEEE Trans. Image Process. 10(4), 573–583 (2001)

    MathSciNet  Google Scholar 

  27. William, T., Touis, R., Egon, C.: Example-based super-resolution. IEEE Comput. Graphics Appl. 56–65 (2002)

  28. Yan, W.-Q., Kankanhalli, M.S.: Detection and removal of lighting and shaking artifacts in home videos. In: Proceedings of ACM Multimedia'02, pp. 107–116. Jan-les-Pins, France (2002)

  29. Farin, G.: Curves and Surfaces for Computer Aided Geometric Design: A Practical Guide, 3rd edn. Academic Press, Boston (1992)

    Google Scholar 

  30. Adams, B., Dorai, C., Venkatesh, S.: Automated film rhythm extraction for scene analysis. In: Proceedings of the IEEE ICME'01, pp. 1192–1195. Tokyo, Japan (2001)

  31. Adams, B., Dorai, C., Venkatesh, S.: Formulating film tempo–-the computational media aesthetics methodology in practice. In: Media Computing: Computational Media Aesthetics, pp. 57–79. Kluwer Academic Publishers, Drodrecht (2002)

    Google Scholar 

  32. Adams, B., Venkatesh, S., Dorai, C.: Finding the beat: an analysis of the rhythmic elements of motion pictures. Int. J. Image Graphics 2(2), 215–245 (2002)

    Google Scholar 

  33. Herbert, Z.: Sight, Sound, Motion: Applied Media Aesthetics, 3rd edn. Wadsworth Publishing Company (1999)

Download references

Author information

Authors and Affiliations

  1. The School of Computing, National University of Singapore, Singapore, 117543, Singapore

    Wei-Qi Yan & Mohan S. Kankanhalli

  2. Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, The Netherlands

    Jun Wang

Authors
  1. Wei-Qi Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohan S. Kankanhalli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohan S. Kankanhalli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yan, WQ., Kankanhalli, M.S. & Wang, J. Analogies based video editing. Multimedia Systems 11, 3–18 (2005). https://doi.org/10.1007/s00530-005-0186-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-005-0186-3

Keywords

Search

Navigation