Reconfigurable video coding framework and decoder reconfiguration instantiation of AVS

https://doi.org/10.1016/j.image.2008.12.002Get rights and content

Abstract

In 2004, a new standardization activity called reconfigurable video coding (RVC) was started by MPEG with the purpose of offering a framework which provides reconfiguration capabilities for standard video coding technology. The essential idea of RVC framework is a dynamic dataflow mechanism of constructing new video codecs by a collection of video coding tools from video tool libraries. With this objective, RVC framework is not restricted to specific coding standard, but defined at coding tools level with interoperability to achieve high flexibility and reusability. Three elements are normative in RVC framework: decoder description (DD), video tool library (VTL) and abstract decoder model (ADM). With these elements, a standard or new decoder is able to be reconfigured in RVC framework. This paper presents the procedure of describing a reconfigured decoder in DD, reusing and exchanging tools from VTLs and initializing ADM in the dataflow formalism of RVC framework. A decoder configuration which can be instantiated as AVS intra decoder configuration or other new decoder configurations in RVC framework is described as an example by using coding tools from China audio video coding standard (AVS) and MPEG series. It is shown that the process mechanism offered by RVC framework is versatile and flexible to achieve high reusability and exchangeability in decoder configurations.

Introduction

New video coding technologies have been developed rapidly over the past two decades. Many standards have been explored to provide different video coding solutions for the market. For instance, H.264/AVC was standardized in May 2003 and audio video coding standard (AVS) was published as Chinese national standard in February 2006. Variety of the coding standards pushes multimedia devices to support multiple codecs, such as MPEG-2, H.264/AVC and AVS, in order to satisfy different needs of industry. However, it is complicated for multimedia devices to support all kinds of codecs. Moreover, with the development of video coding, more coding technologies will be adopted in the future. Those new technologies can be new coding standards or specific coding methods such as interpolation, inverse transform, and so on. It is difficult for multimedia devices which conform to some certain standards to adapt themselves to the new technologies. An obvious case is from IPTV, there clearly exists the transition from one codec to another in a single channel. As the size of video resolution evolves from SD to HD (and may be HD to SHD, in the future), this transition from one codec to another will be a common phenomenon in IPTV. The current usage of multi-codec-in-a-channel is limited in that the transition of one codec to another is strictly scheduled [1]. A simple and robust solution is required to allow more flexible usage of codecs transmission in a single channel.

At the same time, multimedia applications are growing much faster than the standardization of new coding strategies. New requirements from applications challenge the traditional video coding standards. For example, in the present video market, as the video coding standards are defined at codec level and there is no negotiation between them, compressed data conforming to one video coding standard cannot be decoded by decoders of other standards. Thus, as is often the case, video resources from different video coding standards cannot be adequately shared. Potential applications in video coding market should also be cared. A typical example is that, some tools and syntax elements from some standards such as H.264/AVC are not so useful in some actual applications but the decoder has to support them in order to keep itself as a “legal” one. The encoder is also obliged to produce bitstreams conforming to a standard even some syntax elements are never used. If the encoder discards those unnecessary syntax elements to produce a compact bitstream, currently, this bitstream cannot be decoded correctly because the decoder does not know what happens to it unless a dedicated decoder for this encoder is designed. Ref. [4] shows another example which is for low-complexity or high-performance reason, the encoder is possible to employ some new coding tools and needs to communicate with the decoder to guarantee a consistent decoding process. Obviously, negotiation between the encoder and decoder is expected to serve this kind of applications flexibly and dynamically.

For solving those problems, MPEG developed reconfigurable video coding (RVC) based on coding tool level not only to support multiple video codecs but also to allow the interoperability between different video codecs to include improvements and innovations. The essential idea of RVC is to introduce an interoperable model at coding tool level and allow flexible reconfiguration of those coding tools. Current codec level definition of video coding standards restricts the implementations to some profiles of a specific standard which lacks flexibility and does not allow interoperability between different codecs. Additionally, as many of the existing standards share common or similar coding tools, it is more convenient to exploit such commonalities in coding tool level [2]. To achieve this goal, RVC provides a framework allowing a dynamic development, implementation and adoption of standardized video coding solutions with features of higher flexibility and reusability [3]. Reconfiguration of RVC is no longer kept in codec level of a standard. It could be a new configuration of coding tools from different profiles of one standard, a hybrid configuration with coding tools from different standards, and so on. Moreover, RVC supports the introduction of new coding tools in order to speed up the standardization of new technologies and achieve specific design or performance trade-offs to satisfy application constraints. It allows a completely novel configuration with newly developed coding methods or a combination of existing and new coding tools.

The coding tool level definition of RVC framework allows flexible reconfiguration of coding tools to create different codec solutions on-the-fly. Video tool library (VTL) is used in RVC to collect coding tools which is also called functional units (FUs). To satisfy the increasing requirements better and explore more potential needs of industry, RVC framework supports different VTLs, such as MPEG VTL, AVS VTL and other VTLs to improve its capability. As a new standard developed in recent years, AVS can be applied in IPTV, mobile TV and other related applications in video field. Based on RVC framework, AVS VTL is built which includes a collection of AVS coding tools and conformance test for those tools. Participation of AVS in RVC enriches the significance of RVC framework and brings affirmative impulse for video coding field. This paper takes coding tools from AVS and MPEG VTL for example to present the procedure of reconfiguring a decoder in RVC framework and how to explore the reusability and exchangeability for decoder configurations.

In this paper, the reconfiguration formalism of RVC framework which is based on the dataflow model built from coding tools of VTLs is first introduced. It is shown that the process mechanism and management strategy adopted in RVC framework provide great flexibility and reusability for designers to reconfigure new decoding solutions according to application constraints. Then the procedure of constructing decoder configurations in RVC framework is described. First, based on the syntax description for AVS intra decoder, it illustrates how to employ the method defined in RVC framework to describe a reconfigured bitstream flexibly. According to the bitstream syntax description, a dedicated syntax parser for the bitstream can be generated automatically. Secondly, granularity of coding tools from VTLs such as AVS and MPEG VTL is analyzed and a partition method is realized to explore reusability and exchangeability as much as possible for the proposed decoder configuration. Finally, the architecture for the proposed decoder configuration example is constructed by combining the syntax parser and coding tools from AVS and MPEG VTL. An instantiated implementation and simulation of the architecture which serves as an AVS intra decoder configuration are shown to explain the dataflow of this decoder model. It shows that RVC framework is capable of reconfiguring decoders theoretically and practically.

The remaining of the paper is organized as follows: RVC framework is described in Section 2. Section 3 introduces the AVS standard and its coding tools. Section 4 proposes an example to show the procedure of reconfiguring a decoder in RVC framework which includes the decoder description (DD) process, the coding tool partition with flexible reusability and exchangeability, as well as the final architecture for the decoder configuration example. Section 5 presents implementation of the architecture and dataflow model of the decoder configuration example which is instantiated as an AVS intra decoder configuration in RVC simulation environment. Section 6 concludes the paper. Finally, the future work is prospected in Section 7.

Section snippets

RVC framework

RVC offer a flexible mechanism of combing coding tools to reconfigure decoding solutions. RVC framework adopts dataflow process formalism to modular designs which is different from traditional design methods. In order to describe the modular configurations in decoder, as shown in the RVC conceptual diagram (Fig. 1), DD which contains information of bitstream syntax and FU Network is transmitted to decoder together with the encoded video data. In decoder, three elements are normative in RVC

China audio video coding standard

China audio video coding standard which is referred as AVS [13] is a digital audio and video coding standard developed by AVS Workgroup of China, whose video part (AVS part 2) has been adopted as the national standard of China in February 2006. AVS has been developed for several years and can be applied in many video coding applications.

Similar with MPEG-x and H.26x series of standards, AVS also owns a block-based hybrid coding framework. It adopts the spatial and temporal predictions to

Procedure of decoder configuration in RVC framework

The process mechanism of RVC framework introduced in Section 2 owns high flexibility and interoperability for general decoder configurations. In this section, the decoder configuration procedure in RVC framework is described. For the bitstream syntax description, it explains how to simply define a BS schema for a reconfigured bitstream by taking bitstream syntax description of AVS for example. It also depicts how to validate BS schema by BSDL parsers to ensure that the automatically generated

Implementation and simulation for the proposed decoder configuration

All FUs used in the proposed decoder configuration example are specified in RVC–CAL language. In implementation, the proposed decoder configuration example can be instantiated as an AVS intra decoder or other hybrid decoders in RVC framework. In the example shown below, it is configured as an AVS intra decoder by using corresponding coding tools from AVS VTL besides reusing FUs from MPEG VTL. The reconfigured AVS intra decoder is compiled with RVC simulation tool and result of simulation shows

Conclusion

This paper describes the decoder reconfiguration procedure in RVC framework by showing a decoder configuration architecture which successfully combines coding tools from AVS and MPEG VTL to form different decoding solutions. BS schema of AVS intra decoder configuration is detailed to explain how to define the BS schema for a reconfigured bitstream. FUs partition granularity of AVS and MPEG VTL is analyzed to reach high reusability in the proposed decoder configuration architecture. Besides the

The future work

At present, there is still something to be done in RVC proceeding. In the realization of RVC framework, for the automatic generation of syntax parser, an optimized version is in development to deal with more complex cases in H.264/AVC which contains far more new technologies to improve performance. How to implement CABAC is also an emergent issue in RVC to work more efficiently. With the work of RVC proceeds, more complicated implementations of decoder configuration will be synchronically

Acknowledgement

This work was supported in part by the National Science Foundation of China under Grants 60833013 and 60833006.

References (23)

  • Euee S. Jang, Chungku Lee, Study of Application Requirements Related to RVC, ISO/IEC JTC1/WG11 Document M15117,...
  • Euee S. Jang, Jens Ohm, Marco Mattavelli, Whitepaper on Reconfigurable Video Coding (RVC), ISO/IEC JTC1/WG11 Document...
  • Reconfigurable Video Coding Requirements, ISO/IEC JTC1/WG11 Document W8943, SanJose, April...
  • Dandan Ding, Lu Yu, Christophe Lucarz, Marco Mattavelli, Video Decoder Reconfigurations and AVS Extensions in the New...
  • Dandan Ding, Honggang Qi, Lu Yu, RVC Exploration Experiments Report on New 8×8 Inverse Transform and Quantization...
  • Johan Eker, Jörn Janneck, CAL Language Report, ERL Technical Memo UCB/ERL M03/48,...
  • E.S. Jang, K. Asai, C.-J. Tsai, Study of Video Coding Tool Repository, ISO/IEC JTC1/WG11 Document W7329, Poznan, July...
  • Gwo Giun Lee, Euee S. Jang, Marco Mattavelli, Chun-Jen Tsai, Christophe Lucarz, Mickaël Raulet, Dandan Ding, Study Text...
  • Information Technology—MPEG systems technologies—Part 5: Bitstream Syntax Description Language (BSDL), ISO/IEC FDIS...
  • C.-J. Tsai, Suggestions on the Direction of VCTR, ISO/IEC JTC1/WG11 Document M12074, Busan, April...
  • Dandan Ding, Honggang Qi, Lu Yu, Tiejun Huang, Wen Gao, Results of RVC Explore Experiment: Analysis of Function Units...
  • Cited by (7)

    View all citing articles on Scopus

    This work was supported in part by the National Science Foundation of China under Grants 90207005 and the Major State Basic Research Development Program of China (973 Program) under Grants 2009CB320900.

    View full text