Flexible forward error correction codes with application to partial media data recovery
Introduction
Recovery from packet losses is an important problem in packet-switched networking. Traditionally, it has been solved by retransmitting the lost packets (automatic repeat request (ARQ)) or transmitting redundant data that can be used to regenerate lost data blocks (forward error correction (FEC)). In multimedia communications, such as video telephony or streaming, real-time requirements and limited link capacity often discourage the use of retransmissions. In this case, an error-free transmission cannot be guaranteed, and the application has to mask to effect the potential losses by regenerating the blocks of data that have been lost using different interpolation mechanisms. This approach is often referred as error concealment, since it aims to conceal the impact of errors instead of actually recovering fully the errors [1].
In order to facilitate error concealment, FEC can be used to reduce the effective rate of lost data packets. A typical FEC coder is a block coder that takes a block of k source symbols as input and generates n FEC symbols as output (n>k). The code rate R of the code is defined as R=k/n. A data packet can be typically considered as a source symbol, or advanced techniques can use complex partitioning and packetization of data. According to the conventional wisdom, a good FEC code should be designed so that the redundancy overhead required to recover all the missing source symbols is minimized. Therefore, in an ideal case, FEC decoder can regenerate the original k symbols when any k of them out of n has been received. In other words, a random loss of a fraction p of the transmitted symbols can be recovered if p⩽1−R. This can be achieved with minimum distance separable (MDS) codes, such as Reed–Solomon (RS) codes [2]. Unfortunately, if there are more than n−k symbols erased, none of the lost source symbols can be recovered. Multimedia applications can however generally tolerate a small number of packet losses. It would be better to recover at least some of the lost data packets, even if it is not possible to recover all of them.
FEC schemes can be used to blindly recover lost data packets independently on the content they carry. Alternatively, the FEC protection can be computed in order to prioritize classes of packets with joint source-channel coding (JSCC) [3], where the protection can be adjusted to the channel conditions and to the content of the packets. Multimedia data for example typically contain components with different perceptual significance and benefit greatly from differential protection or unequal erasure/error protection (UEP). When conventional MDS codes are used for UEP, separate block codes are needed for data symbols belonging to different priority classes. This may cause serious problems with timing and scheduling, especially with strict delay constraints. In this case, long blocks are required to achieve distinctive protection levels. There is often a mismatch between the priority levels defined by the source encoder, and the flexibility of the UEP solutions based on conventional MDS codes.
In order to overcome the limitations of the conventional channel codes, we present FEC codes that are designed for partial recovery of lost source symbols and we study their performance in media streaming applications. In particular, we focus on Reed–Solomon type of codes with modified generator matrices that offer more flexibility in the FEC design. With this approach, some of the lost source symbols may be recovered even in the cases where MDS codes fail to recover any erasure. In addition, the proposed codes can easily be adjusted for efficient UEP with relatively short source blocks. These features are essential to overcome the abovementioned weaknesses of the conventional MDS codes, especially in delay-sensitive applications like media streaming scenarios, where very long block codes cannot be afforded.
The rest of this paper is organized as follows. In Section 2, the background and relevant related work is discussed. In Section 3, different FEC designs are proposed and studied analytically. In Section 4, the practical benefits of the proposed codes are demonstrated by analyzing their performance in realistic video streaming scenarios. Finally, the conclusions are given in Section 5.
Section snippets
Background and related work
There are several different alternatives to implement generic FEC codes that are usually driven by the characteristics of the target application. In this paper, we focus on systematic linear block codes for erasure channels, where each FEC symbol is a linear combination of source symbols. The simplest possible linear block code is generated by applying the binary exclusive OR (XOR) operation across the binary source symbols. If one of the source symbols is lost, it can be recovered by applying
Preliminaries
We focus now on a generic FEC code design based on RS codes, with a modified generator matrix structure, in order to overcome the all-or-nothing recovery characteristics of conventional codes and their lack of flexibility in UEP design. We focus on relatively short block codes, as we target media streaming applications with delay constraints. The use of MDS codes is thus reasonable in terms of computational complexity. We model the channel as a packet erasure channel with a uniform i.i.d loss
Simulation setup
We analyze now the performance of the proposed sparse codes for equal or unequal error protection and typical media streaming scenarios. We have encoded video sequences in the H.264/AVC format using the H.264/AVC JM reference codec version 13.2 [21]. The encoder includes three basic types of frames: Intra frames (I-frames) that are independent on other frames, and inter frames that use either unidirectional (P-frames) or bidirectional (B-frames) temporal prediction from other frames [22]. A set
Conclusions
In this paper, we have presented sparse FEC codes that are optimized for partial recovery of data in cases where conventional MDS FEC codes fail to recover the lost source packets. They permit to reduce the variations in the residual packet loss rate when the fraction of packet losses per block occasionally exceeds the FEC overhead. We have observed that common short source blocks tend to provide smoother average error correcting performance, at the cost of weaker average performance at low
References (26)
- et al.
Error resilient video coding techniques
IEEE Signal Processing Magazine
(2000) - et al.
Error Control Coding: Fundamentals and Applications
(1983) - et al.
Joint source-channel coding for video communications
- J. Rosenberg, H. Schultzrinne, An RTP Payload Format for Generic Forward Error Correction, IETF RFC 2733,...
- V. Roca, C. Neumann, Design, Evaluation and Comparison of Four Large Block FEC Codecs: LDPC, LDGM, LDGM–Staircase and...
- M. Luby, M. Mitzenmacher, A. Shokrollahi, D. Spielman, V. Stemann, Practical loss-resilient codes, in: Proceedings of...
- et al.
Low-density parity check codes over GF(q)
IEEE Communications Letters
(1998) - et al.
Regular and irregular progressive edge-growth tanner graphs
IEEE Transactions on Information Theory
(2005) Raptor codes
IEEE Transactions on Information Theory
(2006)- A. Kamra, J. Feldman, V. Misra, D. Rubenstein, Growth codes: maximizing sensor network data persistence, in:...
Priority encoding transmission
IEEE Transactions on Information Theory
Cited by (17)
SJA: Server-driven Joint Adaptation of Loss and Bitrate for Multi-Party Realtime Video Streaming
2023, Proceedings - IEEE INFOCOMIvory: Learning Network Adaptive Streaming Codes
2022, 2022 IEEE/ACM 30th International Symposium on Quality of Service, IWQoS 2022Low-latency network-adaptive error control for interactive streaming
2019, MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia
- 1
Part of the work for this paper has been done at Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland. Part of this work was carried out during the tenure of an ERCIM "Alain Bensoussan" Fellowship Programme.
- 2
“Centre for Quantifiable Quality of Service in Communication Systems, Centre of Excellence” appointed by the Research Council of Norway, funded by the Research Council, NTNU and UNINETT.