Video rate control strategies for cloud gaming

https://doi.org/10.1016/j.jvcir.2015.03.012Get rights and content

Highlights

  • We discuss about the features of game players’ attention.

  • We explore the ways to make use of them and improve transmission efficiency.

  • We raise some methods of extracting ROI and key frames from gaming videos.

  • We propose a Macro-block level rate control scheme based on ROI and key frames.

  • Experimental results show that our method outperforms others.

Abstract

Cloud gaming, also called gaming on demand, is a new kind of service that provides real-time video game experience to the players over the Internet. Although cloud gaming services are getting more and more popular recently, its performance is highly limited by the network bandwidth and latency. This work makes use of the unique characteristics of human visual system (HVS) of video game players to improve bandwidth efficiency. In this work, discussions about the characteristics of game players’ HVS are conducted. The discussions can be further extended to all interactive video on demand systems. Then, some schemes of extracting region of interest and key frames from gaming videos are raised. Based on that, a Macro-block level rate control scheme is proposed based on region of interest and scene-change detection. The simulation results show that, under the same bandwidth constraint, the video quality of proposed method outperforms other methods.

Introduction

Cloud gaming is a new popular Internet service that combines the concepts of cloud computing and on-line gaming. It provides the entire gaming experience to the gamers by using the resource of the remote computing servers. Because all the graphics computing and data processing are done in the remote data center, the user’s terminal device is nothing more than a controller (mouse, keyboard or game controller) plus a monitor. The player no longer needs to buy expensive and cutting-edge gaming hardware, like graphics card and big RAM, but still enjoy the latest game. The terminal devices only need broadband Internet connections and the ability to display High Definition (HD) video.

An overview flow chart is shown in Fig. 1. As explained in [1], on the client side, the user controls the game just like on a local device, such as PC, TV or mobile devices. Every time the player performs an operation, such as pressing a key, moving the mouse, or using the controller, the cloud gaming system will send the controlling signals to the remote game servers through Internet. On the server side, the remote servers receive the controlling signals and execute the game programs accordingly. Usually, it involves intensive graphics computation to generate high-quality pictures in real time. Then, the servers will stream the compressed gaming video back to users’ devices. All the gaming program execution, graphic computation and video compression are done on the remote game servers. There will be a continuous control stream from user to server and a continuous video stream from server to user all the time until the session is disconnected.

While it may reduce hardware costs for users, and increase the profit for developers and publishers by reducing the expenditures on retail chains, it also raises a lot of new challenges, especially for the service quality in terms of bandwidth and latency for the underlying network. Table 1 shows a list of Pros and Cons of cloud gaming compared with traditional gaming.

As shown in Table 1, a good gaming system must balance the high performance and good accessibility. Recently, cloud gaming becomes a very hot trend in game industry. Many cloud gaming platforms are getting popular, especially after the leading game console corporations all announced that they will integrate cloud gaming systems into their latest game consoles. However, although they provide very impressive gaming experience, it seems that the only bottleneck that hiders people from using them is the high bandwidth requirement. Thus, solving this problem may be a matter of life and death for this business.

When cloud gaming is based on a network of a relatively low quality condition, e.g. playing cloud games on a wireless mobile device, users still want the gaming experience to be good and smooth as well. In order to provide a decent and stable video streaming quality under a given network condition (or a limited bit rate), rate control (RC) of video coding must be performed. Compared to RC for ordinary videos, RC in clouding gaming have more restrictions, like sensitivity to latency, demand for high image quality in key frames. Although many challenges are faced, many unique characteristics of human visual system (HVS) of video game players can also be exploited to improve bandwidth efficiency. This work focuses on these problems. Since proposed work is the first one to solve this problem, it shows great potential for the development of the cloud gaming industry.

Section snippets

Related work

Since cloud gaming is a newest concept, its related research is not fully conducted yet. To the best of the authors’ knowledge, almost all related researches on this topic are conducted after 2010. By now, most of the papers are discussing the quality of experience (QoE) evaluation scheme, like [2], and measurement of latency, like [3]. Hobfeld et al. [4] pointed out several challenges related to cloud application’s QoE management. However, no one proposed the rate control scheme for cloud

ROI and key frame patterns for gaming video

Since people are more sensitive to the areas where they are interested in, it is reasonable to enhance the region of interest (ROI) while sacrificing the non-ROI regions when the overall coding and transmission resources are limited.

Compared with general videos, encoding cloud gaming videos has three convenient and exceptional features that can be exploited to enhance the performance. First, people are far more concentrative on the ROI while playing games than watching ordinary videos, so the

Rate control for H.264/AVC based video

JVT-G012 [16] is the rate control algorithm recommended by H.264/AVC standard. In JVT-G012, the target bit for each frame is first determined according to the given bandwidth, and the target bit for each MB is determined according to its predicted mean absolute deviation (MAD). The MAD is predicted by a linear model. With the target bit rate, the last step is to determine the QP for each MB using a quadratic rate quantization (R-Q) model. The allocation of bits between MBs is based on the MAD

QoE based evaluation and simulation results

After proposing the strategy, the evaluation and comparison of this strategy need to be conducted. Many of the current cloud gaming papers are about QoE evaluation, which give us the tools to assess the performance of proposed scheme both objectively and subjectively.

Conclusions

In this paper, some strategies for rate control of cloud gaming video are presented. Firstly, the discussions about the characteristics of game players’ HVS and the possible ways to make use of them are conducted. These discussions can be extended into a broader range of applications, including all interactive VOD systems. And a new bit allocation scheme is proposed on MB layer based on ROI. In addition, an objective QoE based quality evaluation method is also proposed. Experimental results

References (19)

  • R. Shea et al.

    Cloud gaming: architecture and performance

    IEEE Network

    (2013)
  • M. Jarschel, D. Schlosser, S. Scheuring, T. Hoßfeld, An evaluation of QoE in cloud gaming based on subjective tests,...
  • K.-T. Chen et al.

    Measuring the latency of cloud gaming systems

  • T. Hobfeld et al.

    Challenges of QoE management for cloud applications

    IEEE Commun. Mag.

    (2012)
  • Y. Liu et al.

    A novel rate control scheme for low delay video communication of H.264/AVC standard

    IEEE Trans. Circ. Syst. Video Technol.

    (2007)
  • Y. Liu, Z.G. Li, Y.C. Soh, Adaptive MAD prediction and refined R-Q model for H.264/AVC rate control, in: Proceedings of...
  • B. Yan et al.

    Joint complexity estimation of I-frame and P-frame for H.264/AVC rate control

    IEEE Trans. Circ. Syst. Video Technol.

    (2012)
  • K. Sun et al.

    Efficient P-frame complexity estimation for frame layer rate control of H.264/AVC

  • C. Lee et al.

    Real-time frame-layer H.264 rate control for scene-transition video at low bit rate

    IEEE Trans. Consumer Electron.

    (2007)
There are more references available in the full text version of this article.

Cited by (15)

View full text