Elsevier

Pattern Recognition Letters

Volume 24, Issue 15, November 2003, Pages 2675-2686
Pattern Recognition Letters

QoS based video delivery with foveation and bandwidth monitoring

https://doi.org/10.1016/S0167-8655(03)00110-7Get rights and content

Abstract

Spatially varying sensing (foveation) was first used as a means for image compression in our past research [IEEE Systems, Man, and Cybernetics Conference Proceedings, 1993, p. 170]. In this report we extend previous work to address the advantages of foveation in improving the performance of MPEG compression over bandwidth limited channels, such as the Internet. Unlike other approaches to foveating MPEG that used multiresolution representations, we use continuously spatially varying resolution and demonstrate that this approach is indeed advantageous over others. Two parameters, scaling and distortion, are used to allow us to adapt MPEG video to various compression ratios depending on the available network resources. Network bandwidth can be optimally monitored following a statistical approach. Experimental results are presented to validate both the bandwidth monitoring algorithm and MPEG foveation.

Introduction

With the increasing usage of the Internet, delivery of multimedia, especially video, is becoming increasingly important. Unfortunately, compression algorithms such as MPEG do not adapt very well to low and more importantly dynamic bandwidth. In this report we present an alternative approach, “foveated” MPEG, which can be adapted to dynamically changing bandwidth in a manner that allows details to be retained in the regions of interest (foveae) in a video sequence. With this differential mechanism, we store different quality profiles on the server, and according to the currently monitored bandwidth and user’s quality requirements, we can adaptively deliver video over the Internet. Compared with traditional Quality-of-Service (QoS) tools that are implemented at the network layer (Ferguson and Huston, 1998; Carter and Crovella, 1996), our method accomplishes QoS delivery on the application level.

The Internet was originally designed to offer only one level of service (best effort) to all its service users. However, as new applications continue to emerge, more and more applications now need to provide some kind of quality guarantees to its users. How to provide this consistent service quality is now an active research area termed QoS (Ferguson and Huston, 1998; Carter and Crovella, 1996; Vogel et al., 1995; Miloucheva, 1995; Pacifici and Stadler, 1995; Gopalakrishna and Parulkar, 1994). One solution to the problem is to allow the application to reserve a certain quality of network services in advance (Vogel et al., 1995; Miloucheva, 1995). Once such a level of service is reserved, the underlying network will commit the corresponding resources needed and guarantee the delivery of such service. Unfortunately, some network architecture and implementations may not support reservation or only support reservation to a limited degree. One alternative approach that has generated a lot of interest in QoS based multimedia delivery application is the so-called network-aware application (Bolliger and Gross, 1998; Bolliger et al., 1999; Wang and Schulzrinne, 1999). These applications do not rely on the underlying network to provide them a guaranteed quality. Instead, they actively monitor the performance variation of the network, and attempt to adjust its resource demands in response. In (Cheng et al., 2001), the authors present a Tele-Learning system that deliver multimedia course materials (including JPEG images) to students over the Internet. The system allows the users to specify QoS parameters (time limit, resolution and quality of image) at the client side, and adapt the resolution and quality of the requested image on the server side to provide best quality image to the user while restricting transmission time within the time limit.

The remainder of this paper is organized as follows: Section 2 describes the variable resolution transform. A method for foveating MPEG is discussed in Section 3. Section 4 describes bandwidth monitoring and QoS based video delivery. Section 5 describes experimental results and analyzes the relationship between the foveation parameters and the compression ratio.

Section snippets

The VR transform

The variable resolution (VR) (Basu et al., 1993; Weiman and Chaikin, 1979) transform used here has two parameters which affect the resulting image: the scaling factor (s) and distortion ratio (δ), which controls the distortion at the edges of the image with respect to the fovea. A high δ value gives a sharply defined fovea with a poorly defined periphery; a small δ value makes the fovea and periphery closer in resolution. Under the VR transform, a pixel with polar coordinates (r,θ) is mapped to

Foveated MPEG

In this paper, the CVR compression techniques are combined with MPEG to produce a hybrid CVR/MPEG video compression format. Fig. 1 illustrates how the input image sequence is passed first through the CVR routines before being further compressed by the MPEG algorithm. The process of decompressing a CVR/MPEG file consists of MPEG decoding followed by the inverse CVR transformation. The software used to implement MPEG (ISO/IEC 11172-2 International Standard, 1993) was not altered. The CVR

Bandwidth monitoring and QoS specification

Two approaches exist to address the problem of accurate bandwidth estimation. In the first approach, the media server maintains a connection for each client currently active. The server sends testing packets periodically to the client and calculates the available bandwidth based on the time to transmit the testing packet. This maintains a history of bandwidth samples. When media delivery request is made to the server, the server estimates the available bandwidth of the future period by using a

Experimental results and analysis

Data was collected to find the relationship between the compression ratio and the scaling and distortion factors. A hierarchy of MPEG clips was constructed using different scaling and distortion factors. According to the current bandwidth and the requirement of the user, one video clip, which adapts to the current bandwidth, is selected from this hierarchy and sent to the user. We will show the distortion values (δ) scaled between 10 and 900 to avoid decimal numbers.

Conclusions

From test results on several standard MPEG video clips, we found that by controlling the scaling and distortion factors we can compress an MPEG video to a desired size to adapt to the available bandwidth on the Internet. Setting up a series of hierarchies of compressed MPEG video, associated with CVR parameters, allows a video server to automatically select the suitable CVR compressed MPEG video for a client to view according to the clients’ requirement and available bandwidth on the network.

Acknowledgements

The authors would like to thank Kevin Wiebe and Lei Chen for their assistance in experiments.

References (15)

  • Basu, A., Sullivan, A., Wiebe, K.J., 1993. Variable resolution teleconferencing, IEEE Systems, Man, and Cybernetics...
  • J. Bolliger et al.

    A framework-based approach to the development of network-aware applications

    IEEE Trans. Software Eng.

    (1998)
  • Bolliger, J., Gross, T., et al., 1999. Bandwidth modeling for network-aware applications. In: Proc. Infocomm’99,...
  • Carter, R.L. , Crovella, M.E., 1996. Measuring Bottleneck Link Speed in Packet-Switched Networks, BU-CS-96-006,...
  • Cheng, I., Basu, A., Zhang, Y., Tripathi, S., 2001. QoS Specification and Adaptive Bandwidth Monitoring for Multimedia...
  • P. Ferguson et al.

    Quality of Service: Delivering QoS on the Internet and in Corporate Networks

    (1998)
  • Gopalakrishna, G., Parulkar, G., 1994. Efficient quality of service in multimedia computer operating systems,...
There are more references available in the full text version of this article.

Cited by (4)

Parts of this research were presented at the IEEE International Conference on Image Processing, Thessaloniki, Greece, October, 2001 and the IEEE International Symposium on Circuits and Systems, Scottsdale, USA, May, 2002.

View full text