Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Automatic Prediction of Perceptual Video Quality: Recent Trends and Research Directions

  • Chapter
High-Quality Visual Experience

Part of the book series: Signals and Communication Technology ((SCT))

  • 1265 Accesses

Abstract

Objective video quality assessment (VQA) refers to evaluation of the quality of a video by an algorithm. The performance of any such VQA algorithm is gaged by how well the algorithmic scores correlate with human perception of quality. Research in the area of VQA has produced a host of full-reference (FR) VQA algorithms. FR VQA algorithms are those in which the algorithm has access to both the original reference video and the distorted video whose quality is being assessed. However, in many cases, the presence of the original reference video is not guaranteed. Hence, even though many FR VQA algorithms have been shown to correlate well with human perception of quality, their utility remains constrained. In this chapter, we analyze recently proposed reduced/no-reference (RR/NR) VQA algorithms. RR VQA algorithms are those in which some information about the reference video and/or the distorting medium is embedded in the video under test. NR VQA algorithms are expected to assess the quality of videos without any knowledge of the reference video or the distorting medium. The utility of RR/NR algorithms has prompted the Video Quality Experts Group (VQEG) to devote resources towards forming a RR/NR test group. In this chapter, we begin by discussing how performance of any VQA algorithm is evaluated. We introduce the popular VQEG Phase-I VQA dataset and comment on its drawbacks. New datasets which allow for objective evaluation of algorithms are then introduced. We then summarize some properties of the human visual system (HVS) that are frequently utilized in developing VQA algorithms. Further, we enumerate the paths that current RR/NR VQA algorithms take in order to evaluate visual quality. We enlist some considerations that VQA algorithms need to consider for HD videos. We then describe exemplar algorithms and elaborate on possible shortcomings of these algorithms. Finally, we suggest possible future research directions in the field of VQA and conclude this chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 179.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 229.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bt-500-11: Methodology for the subjective assessment of the quality of television pictures. International telecommuncation union

    Google Scholar 

  2. Sekuler, R., Blake, R.: Perception. McGraw Hill, New York (2002)

    Google Scholar 

  3. Wandell, B.: Foundations of vision. Sinauer Associates (1995)

    Google Scholar 

  4. Seshadrinathan, K., Bovik, A.C.: Video quality assessment. In: Bovik, A.C. (ed.) The Essential Guide to Video Processing. Academic Press, London (2009)

    Google Scholar 

  5. Moorthy, A.K., Seshadrinathan, K., Bovik, A.C.: Digital Video Quality Assessment Algorithms. Springer, Heidelberg (2009)

    Google Scholar 

  6. Final report from the video quality experts group on the validation of objective quality metrics for video quality assessment, http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseI

  7. Girod, B.: What’s wrong with mean-squared error? In: Watson, A.B. (ed.) Digital images and human vision, pp. 207–220 (1993)

    Google Scholar 

  8. Wang, Z., Bovik, A.C.: Mean squared error: Love it or leave it? - a new look at fidelity measures. IEEE Signal Processing Magazine (2009)

    Google Scholar 

  9. Seshadrinathan, K.: Video quality assessment based on motion models. Ph.D. thesis, The University of Texas at Austin (2008)

    Google Scholar 

  10. Wang, Z., Li, Q.: Video quality assessment using a statistical model of human visual speed perception. Journal of the Optical Society of America 24(12), B61–B69 (2007)

    Article  Google Scholar 

  11. Generic coding of moving pictures and associated audio information - part 2: Video, ITU-T and ISO/IEC JTC 1. ITU-T Recommendation H.262 and ISO/IEC 13 818-2 (MPEG-2) (1994)

    Google Scholar 

  12. Advanced video coding, ISO/IEC 14496-10 and ITU-T Rec. H.264 (2003)

    Google Scholar 

  13. Live wireless video database (2009), http://live.ece.utexas.edu/research/quality/live_wireless_video.html

  14. Live video database (2009), http://live.ece.utexas.edu/research/quality/live_video.html

  15. Seshadrinathan, K., Soundararajan, R., Bovik, A.C., Cormack, L.K.: Study of subjective and objective quality assessment of video. IEEE Transactions on Image Processing (to appear)

    Google Scholar 

  16. Moorthy, A.K., Bovik, A.C.: Wireless video quality assessment: A study of subjective scores and objective algorithms. IEEE Transactions on Circuits and Systems for Video Technology (to appear)

    Google Scholar 

  17. Sheskin, D.: Handbook of parametric and nonparametric statistical procedures. CRC Pr. I Llc., Boca Raton (2004)

    MATH  Google Scholar 

  18. Moorthy, A.K., Bovik, A.C.: A motion-compensated approach to video quality assessment. In: Proc. IEEE Asilomar Conference on Signals, Systems and Computers (2009)

    Google Scholar 

  19. Wang, Z., Lu, L., Bovik, A.: Foveation scalable video coding with automatic fixation selection. IEEE Transactions on Image Processing 12(2), 243 (2003)

    Article  Google Scholar 

  20. Beauchemin, S., Barron, J.: The computation of optical flow. ACM Computing Surveys (CSUR) 27(3), 433–466 (1995)

    Article  Google Scholar 

  21. Richardson, I.: H. 264 and MPEG-4 video compression: video coding for next-generation multimedia. John Wiley & Sons Inc., Chichester (2003)

    Book  Google Scholar 

  22. Winkler, S.: A perceptual distortion metric for digital color video. In: Proc. SPIE, vol. 3644(1), pp. 175–184 (1999)

    Google Scholar 

  23. Sugimoto, O., Kawada, R., Wada, M., Matsumoto, S.: Objective measurement scheme for perceived picture quality degradation caused by MPEG encoding without any reference pictures. In: Proceedings of SPIE, vol. 4310, p. 932 (2000)

    Google Scholar 

  24. MacWilliams, F., Sloane, N.: Pseudo-random sequences and arrays. Proceedings of the IEEE 64(12), 1715–1729 (1976)

    Article  MathSciNet  Google Scholar 

  25. Campisi, P., Carli, M., Giunta, G., Neri, A.: Blind quality assessment system for multimedia communications using tracing watermarking. IEEE Transactions on Signal Processing 51(4), 996–1002 (2003)

    Article  MathSciNet  Google Scholar 

  26. Fu-zheng, Y., Xin-dai, W., Yi-lin, C., Shuai, W.: A no-reference video quality assessment method based on digital watermark. In: 14th IEEE Proceedings on Personal, Indoor and Mobile Radio Communications, PIMRC 2003, vol. 3 (2003)

    Google Scholar 

  27. Farias, M., Carli, M., Neri, A., Mitra, S.: Video quality assessment based on data hiding driven by optical flow information. In: Proceedings of the SPIE Human Vision and Electronic Imaging IX, San Jose, CA, USA, pp. 190–200 (2004)

    Google Scholar 

  28. Carli, M., Farias, M., Gelasca, E., Tedesco, R., Neri, A.: Quality assessment using data hiding on perceptually important areas. In: IEEE International Conference on Image Processing (2005)

    Google Scholar 

  29. Wolf, S., Pinson, M.: Video quality measurement techniques. National Telecommunications and Information Administration (NTIA) Report 02-392 (2002)

    Google Scholar 

  30. Wolf, S., Pinson, M.: Low bandwidth reduced reference video quality monitoring system. In: First International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ, USA (2005)

    Google Scholar 

  31. Oelbaum, T., Diepold, K.: Building a reduced reference video quality metric with very low overhead using multivariate data analysis. In: International Conference on Cybernetics and Information Technologies, Systems and Applications, CITSA 2007 (2007)

    Google Scholar 

  32. Oelbaum, T., Diepold, K.: A reduced reference video quality metric for avc/h.264. In: Proc. European Signal Processing Conference, pp. 1265–1269 (2007)

    Google Scholar 

  33. Hair, J.: Multivariate data analysis. Prentice Hall, Englewood Cliffs (2006)

    Google Scholar 

  34. Haykin, S.: Neural networks: a comprehensive foundation. Prentice Hall, Englewood Cliffs (2008)

    Google Scholar 

  35. Le Callet, P., Viard-Gaudin, C., Barba, D.: A convolutional neural network approach for objective video quality assessment. IEEE Transactions on Neural Networks 17(5), 1316 (2006)

    Article  Google Scholar 

  36. Meng, F., Jiang, X., Sun, H., Yang, S.: Objective Perceptual Video Quality Measurement using a Foveation-Based Reduced Reference Algorithm. In: IEEE International Conference on Multimedia and Expo., pp. 308–311 (2007)

    Google Scholar 

  37. Wang, Z., Wu, G., Sheikh, H., Simoncelli, E., Yang, E., Bovik, A.: Quality-aware images. IEEE Transactions on Image Processing 15(6), 1680–1689 (2006)

    Article  Google Scholar 

  38. Hiremath, B., Li, Q., Wang, Z.: Quality-aware video. In: IEEE International Conference on Image Processing, ICIP 2007, vol. 3 (2007)

    Google Scholar 

  39. Simoncelli, E., Freeman, W., Adelson, E., Heeger, D.: Shiftable multiscale transforms. IEEE Transactions on Information Theory 38(2), 587–607 (1992)

    Article  MathSciNet  Google Scholar 

  40. Cover, T., Thomas, J.: Elements of information theory. Wiley-Interscience, Hoboken (2006)

    MATH  Google Scholar 

  41. Yamada, T., Miyamoto, Y., Serizawa, M., Harasaki, H.: Reduced-reference based video quality-metrics using representative-luminance values. In: Third International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ, USA (2007)

    Google Scholar 

  42. Gunawan, I., Ghanbari, M.: Reduced-reference video quality assessment using discriminative local harmonic strength with motion consideration. IEEE Transactions on Circuits and Systems for Video Technology 18(1), 71–83 (2008)

    Article  Google Scholar 

  43. Gonzalez, R., Woods, R.: Digital image processing. Prentice-Hall, Inc., Upper Saddle River (2002)

    Google Scholar 

  44. Oppenheim, A., Schafer, R.: Discrete-time signal processing. Prentice-Hall, Inc., Upper Saddle River (1989)

    MATH  Google Scholar 

  45. Valenzise, G., Naccari, M., Tagliasacchi, M., Tubaro, S.: Reduced-reference estimation of channel-induced video distortion using distributed source coding. In: Proceeding of the 16th ACM international conference on Multimedia (2008)

    Google Scholar 

  46. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From error measurement to structural similarity. IEEE Signal Processing Letters 13(4), 600–612 (2004)

    Google Scholar 

  47. Wang, Z., Lu, L., Bovik, A.C.: Video quality assesssment based on structural distortion measurement. Signal Processing: Image communication (2), 121–132 (2004)

    Google Scholar 

  48. Albonico, A., Valenzise, G., Naccari, M., Tagliasacchi, M., Tubaro, S.: A Reduced-Reference Video Structural Similarity metric based on no-reference estimation of channel-induced distortion. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP (2009)

    Google Scholar 

  49. Naccari, M., Tagliasacchi, M., Pereira, F., Tubaro, S.: No-reference modeling of the channel induced distortion at the decoder for H. 264/AVC video coding. In: Proceedings of the International Conference on Image Processing, San Diego, CA, USA (2008)

    Google Scholar 

  50. Tan, K., Ghanbari, M.: Blockiness detection for MPEG2-coded video. IEEE Signal Processing Letters 7(8), 213–215 (2000)

    Article  Google Scholar 

  51. Vlachos, T.: Detection of blocking artifacts in compressed video. Electronics Letters 36(13), 1106–1108 (2000)

    Article  Google Scholar 

  52. Winkler, S., Sharma, A., McNally, D.: Perceptual video quality and blockiness metrics for multimedia streaming applications. In: Proceedings of the International Symposium on Wireless Personal Multimedia Communications, pp. 547–552 (2001)

    Google Scholar 

  53. Suthaharan, S.: Perceptual quality metric for digital video coding. Electronics Letters 39(5), 431–433 (2003)

    Article  Google Scholar 

  54. Muijs, R., Kirenko, I.: A no-reference blocking artifact measure for adaptive video processing. In: Proceedings of the 13th European Signal Processing Conference, EUSIPCO 2005 (2005)

    Google Scholar 

  55. Caviedes, J., Oberti, F.: No-reference quality metric for degraded and enhanced video. In: Proceedings of SPIE, vol. 5150, p. 621 (2003)

    Google Scholar 

  56. Babu, R., Bopardikar, A., Perkis, A., Hillestad, O.: No-reference metrics for video streaming applications. In: International Packet Video Workshop (2004)

    Google Scholar 

  57. Farias, M., Mitra, S.: No-reference video quality metric based on artifact measurements. In: IEEE International Conference on Image Processing, vol. 3, pp. 141–144 (2005)

    Google Scholar 

  58. Massidda, F., Giusto, D., Perra, C.: No reference video quality estimation based on human visual system for 2.5/3G devices. In: Proceedings of SPIE, vol. 5666, p. 168 (2005)

    Google Scholar 

  59. Marziliano, P., Dufaux, F., Winkler, S., Ebrahimi, T.: Perceptual blur and ringing metrics: Application to JPEG 2000. Signal Processing: Image Communication 19(2), 163–172 (2004)

    Article  Google Scholar 

  60. Dosselmann, R., Yang, X.: A Prototype No-Reference Video Quality System. In: Fourth Canadian Conference on Computer and Robot Vision, CRV 2007, pp. 411–417 (2007)

    Google Scholar 

  61. Kawayoke, Y., Horita, Y.: NR objective continuous video quality assessment model based on frame quality measure. In: 15th IEEE International Conference on Image Processing, ICIP 2008, pp. 385–388 (2008)

    Google Scholar 

  62. Moorthy, A.K., Bovik, A.C.: Visual importance pooling for image quality assessment. IEEE Journal of Selected Topics in Signal Processing, Issue on Visual Media Quality Assessment 3(2), 193–201 (2009)

    Article  Google Scholar 

  63. Yang, F., Wan, S., Chang, Y., Wu, H.: A novel objective no-reference metric for digital video quality assessment. IEEE Signal processing letters 12(10), 685–688 (2005)

    Article  Google Scholar 

  64. Lu, J.: Image analysis for video artifact estimation and measurement. In: Proceedings of SPIE, vol. 4301, p. 166 (2001)

    Google Scholar 

  65. Pastrana-Vidal, R., Gicquel, J.: Automatic quality assessment of video fluidity impairments using a no-reference metric. In: Proc. of Int. Workshop on Video Processing and Quality Metrics for Consumer Electronics (2006)

    Google Scholar 

  66. Pastrana-Vidal, R., Gicquel, J., Colomes, C., Cherifi, H.: Sporadic frame dropping impact on quality perception. In: Proceedings of SPIE, vol. 5292, p. 182 (2004)

    Google Scholar 

  67. Pastrana-Vidal, R., Gicquel, J., Colomes, C., Cherifi, H.: Frame dropping effects on user quality perception. In: 5th International Workshop on Image Analysis for Multimedia Interactive Services (2004)

    Google Scholar 

  68. Lu, Z., Lin, W., Seng, B., Kato, S., Ong, E., Yao, S.: Perceptual Quality Evaluation on Periodic Frame-Dropping Video. In: Proc. of IEEE Conference on Image Processing, pp. 433–436 (2007)

    Google Scholar 

  69. Pastrana-Vidal, R., Gicquel, J.: A no-reference video quality metric based on a human assessment model. In: Third International Workshop on Video Processing and Quality Metrics for Consumer Electronics, VPQM, vol. 7, pp. 25–26 (2007)

    Google Scholar 

  70. Yang, K., Guest, C., El-Maleh, K., Das, P.: Perceptual temporal quality metric for compressed video. IEEE Transactions on Multimedia 9(7), 1528–1535 (2007)

    Article  Google Scholar 

  71. Cotsaces, C., Nikolaidis, N., Pitas, I.: Video shot detection and condensed representation. a review. IEEE signal processing magazine 23(2), 28–37 (2006)

    Article  Google Scholar 

  72. Yamada, T., Miyamoto, Y., Serizawa, M.: No-reference video quality estimation based on error-concealment effectiveness. In: Packet Video 2007, pp. 288–293 (2007)

    Google Scholar 

  73. Keimel, C., Oelbaum, T., Diepold, K.: No-Reference Video Quality Evaluation for High-Definition Video. In: Proceedings of the International Conference on Image Processing, San Diego, CA, USA (2009)

    Google Scholar 

  74. Ong, E., Wu, S., Loke, M., Rahardja, S., Tay, J., Tan, C., Huang, L.: Video quality monitoring of streamed videos. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1153–1156 (2009)

    Google Scholar 

  75. Knee, M.: A single-ended picture quality measure for MPEG-2. In: Proc. International Broadcasting Convention, pp. 7–12 (2000)

    Google Scholar 

  76. Leontaris, A., Reibman, A.: Comparison of blocking and blurring metrics for video compression. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2005, vol. 2 (2005)

    Google Scholar 

  77. Leontaris, A., Cosman, P.C., Reibman, A.: Quality evaluation of motion-compensated edge artifacts in compressed video. IEEE Transactions on Image Processing 16(4), 943–956 (2007)

    Article  MathSciNet  Google Scholar 

  78. Hands, D., Bourret, A., Bayart, D.: Video QoS enhancement using perceptual quality metrics. BT Technology Journal 23(2), 208–216 (2005)

    Article  Google Scholar 

  79. Kanumuri, S., Cosman, P., Reibman, A., Vaishampayan, V.: Modeling packet-loss visibility in MPEG-2 video. IEEE transactions on Multimedia 8(2), 341–355 (2006)

    Article  Google Scholar 

  80. Yuen, M., Wu, H.: A survey of hybrid MC/DPCM/DCT video coding distortions. Signal Processing 70(3), 247–278 (1998)

    Article  MATH  Google Scholar 

  81. Wang, Z., Simoncelli, E.P.: Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities. Journal of Vision 8(12), 1–13 (2008)

    Article  MATH  Google Scholar 

  82. Charrier, C., Knoblauch, K., Moorthy, A.K., Bovik, A.C., Maloney, L.T.: Comparison of image quality assessment algorithms on compressed images. In: SPIE conference on Image quality and System Performance (to appear, 2010)

    Google Scholar 

  83. Charrier, C., Maloney, L.T., Cheri, H., Knoblauch, K.: Maximum likelihood difference scaling of image quality in compression-degraded images. Journal of the Optical Society of America 24(11), 3418–3426 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, 78712, USA

    Anush K. Moorthy & Alan C. Bovik

Authors
  1. Anush K. Moorthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alan C. Bovik
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Surrey, Guildford, UK

    Marta Mrak

  2. University of Zagreb, Zagreb, Croatia

    Mislav Grgic

  3. Swiss Federal Institute of Technology, Lausanne, Switzerland

    Murat Kunt

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Moorthy, A.K., Bovik, A.C. (2010). Automatic Prediction of Perceptual Video Quality: Recent Trends and Research Directions. In: Mrak, M., Grgic, M., Kunt, M. (eds) High-Quality Visual Experience. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12802-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12802-8_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12801-1

  • Online ISBN: 978-3-642-12802-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation