Video rate control strategies for cloud gaming
Introduction
Cloud gaming is a new popular Internet service that combines the concepts of cloud computing and on-line gaming. It provides the entire gaming experience to the gamers by using the resource of the remote computing servers. Because all the graphics computing and data processing are done in the remote data center, the user’s terminal device is nothing more than a controller (mouse, keyboard or game controller) plus a monitor. The player no longer needs to buy expensive and cutting-edge gaming hardware, like graphics card and big RAM, but still enjoy the latest game. The terminal devices only need broadband Internet connections and the ability to display High Definition (HD) video.
An overview flow chart is shown in Fig. 1. As explained in [1], on the client side, the user controls the game just like on a local device, such as PC, TV or mobile devices. Every time the player performs an operation, such as pressing a key, moving the mouse, or using the controller, the cloud gaming system will send the controlling signals to the remote game servers through Internet. On the server side, the remote servers receive the controlling signals and execute the game programs accordingly. Usually, it involves intensive graphics computation to generate high-quality pictures in real time. Then, the servers will stream the compressed gaming video back to users’ devices. All the gaming program execution, graphic computation and video compression are done on the remote game servers. There will be a continuous control stream from user to server and a continuous video stream from server to user all the time until the session is disconnected.
While it may reduce hardware costs for users, and increase the profit for developers and publishers by reducing the expenditures on retail chains, it also raises a lot of new challenges, especially for the service quality in terms of bandwidth and latency for the underlying network. Table 1 shows a list of Pros and Cons of cloud gaming compared with traditional gaming.
As shown in Table 1, a good gaming system must balance the high performance and good accessibility. Recently, cloud gaming becomes a very hot trend in game industry. Many cloud gaming platforms are getting popular, especially after the leading game console corporations all announced that they will integrate cloud gaming systems into their latest game consoles. However, although they provide very impressive gaming experience, it seems that the only bottleneck that hiders people from using them is the high bandwidth requirement. Thus, solving this problem may be a matter of life and death for this business.
When cloud gaming is based on a network of a relatively low quality condition, e.g. playing cloud games on a wireless mobile device, users still want the gaming experience to be good and smooth as well. In order to provide a decent and stable video streaming quality under a given network condition (or a limited bit rate), rate control (RC) of video coding must be performed. Compared to RC for ordinary videos, RC in clouding gaming have more restrictions, like sensitivity to latency, demand for high image quality in key frames. Although many challenges are faced, many unique characteristics of human visual system (HVS) of video game players can also be exploited to improve bandwidth efficiency. This work focuses on these problems. Since proposed work is the first one to solve this problem, it shows great potential for the development of the cloud gaming industry.
Section snippets
Related work
Since cloud gaming is a newest concept, its related research is not fully conducted yet. To the best of the authors’ knowledge, almost all related researches on this topic are conducted after 2010. By now, most of the papers are discussing the quality of experience (QoE) evaluation scheme, like [2], and measurement of latency, like [3]. Hobfeld et al. [4] pointed out several challenges related to cloud application’s QoE management. However, no one proposed the rate control scheme for cloud
ROI and key frame patterns for gaming video
Since people are more sensitive to the areas where they are interested in, it is reasonable to enhance the region of interest (ROI) while sacrificing the non-ROI regions when the overall coding and transmission resources are limited.
Compared with general videos, encoding cloud gaming videos has three convenient and exceptional features that can be exploited to enhance the performance. First, people are far more concentrative on the ROI while playing games than watching ordinary videos, so the
Rate control for H.264/AVC based video
JVT-G012 [16] is the rate control algorithm recommended by H.264/AVC standard. In JVT-G012, the target bit for each frame is first determined according to the given bandwidth, and the target bit for each MB is determined according to its predicted mean absolute deviation (MAD). The MAD is predicted by a linear model. With the target bit rate, the last step is to determine the QP for each MB using a quadratic rate quantization (R-Q) model. The allocation of bits between MBs is based on the MAD
QoE based evaluation and simulation results
After proposing the strategy, the evaluation and comparison of this strategy need to be conducted. Many of the current cloud gaming papers are about QoE evaluation, which give us the tools to assess the performance of proposed scheme both objectively and subjectively.
Conclusions
In this paper, some strategies for rate control of cloud gaming video are presented. Firstly, the discussions about the characteristics of game players’ HVS and the possible ways to make use of them are conducted. These discussions can be extended into a broader range of applications, including all interactive VOD systems. And a new bit allocation scheme is proposed on MB layer based on ROI. In addition, an objective QoE based quality evaluation method is also proposed. Experimental results
References (19)
- et al.
Cloud gaming: architecture and performance
IEEE Network
(2013) - M. Jarschel, D. Schlosser, S. Scheuring, T. Hoßfeld, An evaluation of QoE in cloud gaming based on subjective tests,...
- et al.
Measuring the latency of cloud gaming systems
- et al.
Challenges of QoE management for cloud applications
IEEE Commun. Mag.
(2012) - et al.
A novel rate control scheme for low delay video communication of H.264/AVC standard
IEEE Trans. Circ. Syst. Video Technol.
(2007) - Y. Liu, Z.G. Li, Y.C. Soh, Adaptive MAD prediction and refined R-Q model for H.264/AVC rate control, in: Proceedings of...
- et al.
Joint complexity estimation of I-frame and P-frame for H.264/AVC rate control
IEEE Trans. Circ. Syst. Video Technol.
(2012) - et al.
Efficient P-frame complexity estimation for frame layer rate control of H.264/AVC
- et al.
Real-time frame-layer H.264 rate control for scene-transition video at low bit rate
IEEE Trans. Consumer Electron.
(2007)
Cited by (15)
An Action-Aware Combat Model for Efficient Video Compression of Massively Multiplayer Online Role-playing Games on Cloud Gaming Platforms
2021, IEEE 23rd International Workshop on Multimedia Signal Processing, MMSP 2021CloudyGame: Enabling cloud gaming on the edge with dynamic asset streaming and shared game instances
2020, Multimedia Tools and ApplicationsMulti-objective particle swarm optimization for roi based video coding
2019, 1st ACM International Conference on Multimedia in Asia, MMAsia 2019Are we ready to play in the cloud? Developing new quality certifications to tackle challenges of cloud gaming services
2019, Proceedings - 21st IEEE Conference on Business Informatics, CBI 2019Content-aware video encoding for cloud gaming
2019, Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019A Literature Review - Firm Investment on Cloud as Efficient & Effective Technology
2019, Journal of Physics: Conference Series