Abstract
The concept of regions of interest (ROIs) within a video sequence is useful for many application scenarios. This paper concentrates on the exploitation of ROI coding within the first version of the H.264/AVC specification, for which it was already shown in literature that the flexible macroblock ordering (FMO) tool can be used to achieve ROIs in H.264/AVC video streams. We extend the existing methods with two approaches in order to better match the denotation of ROI scalability. The first approach allows to change the size of the output video pane while the second approach makes it possible to select an ROI at run time without the need for an encoder to provide that specific ROI in the bitstream. It is shown that both approaches allow for real-time adaptation of H.264/AVC bitstreams. Measurements also show that significant bit rate savings can be achieved when performing ROI-based adaptation, that the decoding speed is positively affected, and that the coding overhead can be controlled.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Note that a PPS was already necessary at those points in time.
Note that the speed in terms of slices per second would be comparable.
References
Applications and requirements for scalable video coding. MPEG-document ISO/IEC JTC1/SC29/WG11 N6880, Moving Picture Experts Group (MPEG), Hongkong, China, January 2005. Available on http://www.chiariglione.org/mpeg/working_documents/mpeg-04/svc/requirements.zip
Cimprich, P.: Streaming transformations for XML (STX) version 1.0 working draft. Available on http://stx.sourceforge.net/documents/spec-stx-20040701.html (2004)
De Neve, W., Lerouge, S., Lambert, P., Van de Walle, R.: A performance evaluation of MPEG-21 BSDL in the context of H.264/AVC. In: Proceedings of SPIE annual meeting 2004: Signal and Image Processing and Sensors, vol. 5558, pp. 555–566. Denver, CO, USA (2004)
De Neve, W., Van Deursen, D., De Schrijver, D., Lerouge, S., De Wolf, K., Van de Walle R.: BFlavor: a harmonized approach to media resource adaptation, inspired by MPEG-21 BSDL and XFlavor. EURASIP Signal Process. Image Commun. 21(10), 862–889 (2006)
De Schrijver, D., Van Lancker, W., Van de Walle, R.: Performance of a scalable bitstream adaptation process based on high level XML descriptions. In: Proceedings of the 2005 Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2005), p. 4 on CD–rom, Montreux, Switzerland (2005)
De Sutter, R.: Automated video adaptation based on time-varying context parameters. Dissertation, Universiteit Gent (2006)
Ichimura, D., Honda, Y., Sun, H., Lee, M., Shen, S.: A tool for interactive ROI scalability. JVT-document JVT-Q020, joint video team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG16/Q.6, Nice, France. Available on http://ftp3.itu.int/av-arch/jvt-site (2005)
JOOST: Joost is oli’s original streaming transformer. http://joost.sourceforge.net/
JVT/AVC reference software. http://iphome.hhi.de/suehring/tml/download/
Lambert, P., De Neve, W., De Schrijver, D., Dhondt, Y., Van de Walle R.: Using placeholder slices and MPEG-21 BSDL for ROI extraction in H.264/AVC FMO-encoded bitstream. In: Proceedings of SIGMAP 2006, pp. 9–16. Setúbal, Portugal (2006)
Lambert, P., De Neve, W., Dhondt, Y., Van de Walle, R.: Flexible macroblock ordering in H.264/AVC. J. Vis. Commun. Image Represent. 17, 358–375 (2006)
Lambert, P., De Schrijver, D., Van Deursen, D., De Neve, W., Dhondt, Y., Van de Walle R.: A real-time content adaptation framework for exploiting ROI scalability in H.264/AVC. Lecture Notes in Computer Science, Advanced Concepts for Intelligent Vision Systems (ACIVS 2006), pp. 442–453 (2006)
Li W.: Overview of fine granularity scalability in MPEG-4 video standard. IEEE Trans. Circuits Syst Video Technol 11(3), 301–317 (2001)
Reichel, J., Schwarz, H., Wien, M.: Joint scalable video model JSVM-7. JVT-document JVT-T201, Joint Video Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG16/Q.6, Klagenfurt, Austria. Available on http://ftp3.itu.int/av-arch/jvt-site (2006)
Taubman, D.S., Marcellin, M.W.: JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer, Dordrecht (2002)
Thang, T.C., Kim, D., Bae, T.M., Kang, J.W., Ro, Y.M., Kim, J.-G.: Show case of ROI extraction using scalability information SEI message. JVT-document JVT-Q077, Joint Video Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG16/Q.6, Nice, France. Available on http://ftp3.itu.int/av-arch/jvt-site (2005)
Van Deursen, D., De Schrijver, D., De Neve, W., Van de Walle, R.: A real-time XML-based adaptation system for scalable video formats. Lecture Notes in Computer Science, Advances in Multimedia Information Processing, PCM 2006, 7th Pacific-Rim Conference on Multimedia, vol. 4261, pp. 339–348 (2006)
Yin, P., Boyce, J., Pandit, P.: FMO and ROI scalability. JVT-document JVT-Q029, Joint Video Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG16/Q.6, Nice, France. Available on http://ftp3.itu.int/av-arch/jvt-site (2005)
Acknowledgments
The research activities as described in this paper were funded by Ghent University, the Interdisciplinary Institute for Broadband Technology (IBBT), the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT), the Fund for Scientific Research-Flanders (FWO-Flanders), and the European Union.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lambert, P., Van de Walle, R. Real-time interactive regions of interest in H.264/AVC. J Real-Time Image Proc 4, 67–77 (2009). https://doi.org/10.1007/s11554-008-0102-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-008-0102-0