ABSTRACT
Tile-based approach is widely adopted in adaptive 360\textdegree~video streaming systems. Existing QoE-driven streaming approaches usually obtain the tile selection and adjust the bitrate based on the viewport prediction with a fixed tiling, which fail to consider the unstable prediction performance. However, varying the tiling of the video can produce different number of tiles with different sizes, and thus can have distinct impacts on error tolerance for viewport prediction and on decoding complexity for resource-constrained mobile client. In this work, we introduce adaptive tiling into the conventional bitrate adaptation for mobile 360degree~video streaming. We first analyze the impacts of tilings on tile selection and decoding time, which verify the benefit of tiling adaptation in various practical aspects. We then formulate the QoE optimization problem for adaptive tiling and bitrate streaming and discuss the design details of our adaptation algorithm, which can adapt to the performance of viewport prediction and the decoding capabilities of mobile clients in addition to the conventional influencing factors. Finally, the superiority of our proposed approach compared with the state-of-the-art methods is evaluated through extensive trace-driven simulations.
- Accessed: 2021. FFmpeg. https://ffmpeg.org/.Google Scholar
- Accessed: 2021. Kvazaar endocer. https://github.com/ultravideo/kvazaar.Google Scholar
- Accessed: 2021. MediaCodec API. https://developer.android.com/reference/ android/media/MediaCodec.Google Scholar
- Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: auto-tuning video ABR algorithms to network conditions. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 44--58. Google ScholarDigital Library
- Ghufran Baig, Jian He, Mubashir Adnan Qureshi, Lili Qiu, Guohai Chen, Peng Chen, and Yinliang Hu. 2019. Jigsaw: Robust live 4k video streaming. In Proceedings of the 25th Annual International Conference on Mobile Computing and Networking. ACM, 1--16. Google ScholarDigital Library
- Lovish Chopra, Sarthak Chakraborty, Abhijit Mondal, and Sandip Chakraborty. 2021. PARIMA: Viewport Adaptive 360-Degree Video Streaming. In Proceedings of the Web Conference 2021. 2379--2391. Google ScholarDigital Library
- Erwan J David, Jesús Gutiérrez, Antoine Coutrot, Matthieu Perreira Da Silva, and Patrick Le Callet. 2018. A dataset of head and eye movements for 360textdegree videos. In Proceedings of the 9th ACM Multimedia Systems Conference. ACM, 432--437. Google ScholarDigital Library
- Yu Guan, Chengyuan Zheng, Xinggong Zhang, Zongming Guo, and Junchen Jiang. 2019. Pano: Optimizing 360 video streaming with a better understanding of quality perception. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 394--407. Google ScholarDigital Library
- Jian He, Mubashir Adnan Qureshi, Lili Qiu, Jin Li, Feng Li, and Lei Han. 2018. Rubiks: Practical 360-Degree Streaming for Smartphones. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 482--494. Google ScholarDigital Library
- Yifei Huang, Minjie Cai, Zhenqiang Li, and Yoichi Sato. 2018. Predicting gaze in egocentric video by learning task-dependent attention transition. In Proceedings of the European Conference on Computer Vision (ECCV). 754--769.Google ScholarDigital Library
- Junchen Jiang, Vyas Sekar, and Hui Zhang. 2014. Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. IEEE/ACM Transactions on Networking (ToN), Vol. 22, 1 (2014), 326--340. Google ScholarDigital Library
- Nan Jiang, Yao Liu, Tian Guo, Wenyao Xu, Viswanathan Swaminathan, Lisong Xu, and Sheng Wei. 2020. QuRate: power-efficient mobile immersive video streaming. In Proceedings of the 11th ACM Multimedia Systems Conference. ACM, 99--111. Google ScholarDigital Library
- Nan Jiang, Viswanathan Swaminathan, and Sheng Wei. 2017. Power Evaluation of 360 VR Video Streaming on Head Mounted Display Devices. In Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, 55--60. Google ScholarDigital Library
- Wen-Chih Lo, Ching-Ling Fan, Jean Lee, Chun-Ying Huang, Kuan-Ta Chen, and Cheng-Hsin Hsu. 2017. 360 video viewing dataset in head-mounted virtual reality. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 211--216. Google ScholarDigital Library
- Haoran Lv, Qin Yang, Chenglin Li, Wenrui Dai, Junni Zou, and Hongkai Xiong. 2020. SalGCN: Saliency Prediction for 360-Degree Images Based on Spherical Graph Convolutional Networks. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 682--690. Google ScholarDigital Library
- Saeed Mahmoudpour and Peter Schelkens. 2020. Omnidirectional Video Quality Index Accounting for Judder. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, 1 (2020), 61--75.Google ScholarDigital Library
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 197--210. Google ScholarDigital Library
- Stefano Petrangeli, Viswanathan Swaminathan, Mohammad Hosseini, and Filip De Turck. 2017. An http/2-based adaptive streaming framework for 360 virtual reality videos. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 306--314. Google ScholarDigital Library
- Feng Qian, Bo Han, Qingyang Xiao, and Vijay Gopalakrishnan. 2018. Flare: Practical viewport-adaptive 360-degree video streaming for mobile devices. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. ACM, 99--114. Google ScholarDigital Library
- Giuseppe Ribezzo, Luca De Cicco, Vittorio Palmisano, and Saverio Mascolo. 2020. TAPAS-360: A Tool for the Design and Experimental Evaluation of 360 Video Streaming Systems. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 4477--4480. Google ScholarDigital Library
- Jihoon Ryoo, Kiwon Yun, Dimitris Samaras, Samir R Das, and Gregory Zelinsky. 2016. Design and evaluation of a foveated video streaming service for commodity client devices. In Proceedings of the 7th ACM Multimedia Systems Conference. ACM, 1--11. Google ScholarDigital Library
- Shu Shi, Varun Gupta, Michael Hwang, and Rittwik Jana. 2019. Mobile VR on edge cloud: a latency-driven design. In Proceedings of the 10th ACM Multimedia Systems Conference. ACM, 222--231. Google ScholarDigital Library
- Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, and Gordon Wetzstein. 2018. Saliency in VR: How do people explore virtual environments? IEEE Transactions on Visualization and Computer Graphics, Vol. 24, 4 (2018), 1633--1642. Google ScholarDigital Library
- Yi Sun, Xiaoqi Yin, Junchen Jiang, Vyas Sekar, Fuyuan Lin, Nanshu Wang, Tao Liu, and Bruno Sinopoli. 2016. CS2P: Improving video bitrate selection and adaptation with data-driven throughput prediction. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 272--285. Google ScholarDigital Library
- Jeroen Van Der Hooft, Stefano Petrangeli, Tim Wauters, Rafael Huysegems, Patrice Rondao Alface, Tom Bostoen, and Filip De Turck. 2016. HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks. IEEE Communications Letters, Vol. 20, 11 (2016), 2177--2180.Google ScholarCross Ref
- Shuoqian Wang, Xiaoyang Zhang, Mengbai Xiao, Kenneth Chiu, and Yao Liu. 2020. SphericRTC: A System for Content-Adaptive Real-Time 360-Degree Video Communication. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 3595--3603. Google ScholarDigital Library
- Chenglei Wu, Zhihao Tan, Zhi Wang, and Shiqiang Yang. 2017. A dataset for exploring user behaviors in VR spherical video streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 193--198. Google ScholarDigital Library
- Chenglei Wu, Ruixiao Zhang, Zhi Wang, and Lifeng Sun. 2020. A Spherical Convolution Approach for Learning Long Term Viewport Prediction in 360 Immersive Video. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 14003--14040.Google ScholarCross Ref
- Mengbai Xiao, Chao Zhou, Yao Liu, and Songqing Chen. 2017. Optile: Toward optimal tiling in 360-degree video streaming. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 708--716. Google ScholarDigital Library
- Lan Xie, Zhimin Xu, Yixuan Ban, Xinggong Zhang, and Zongming Guo. 2017. 360probdash: Improving qoe of 360 video streaming using tile-based http adaptive streaming. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 315--323. Google ScholarDigital Library
- Lan Xie, Xinggong Zhang, and Zongming Guo. 2018. Cls: A cross-user learning based system for improving qoe in 360-degree video adaptive streaming. In Proceedings of the 26th ACM International Conference on Multimedia. ACM, 564--572. Google ScholarDigital Library
- Yanyu Xu, Yanbing Dong, Junru Wu, Zhengzhong Sun, Zhiru Shi, Jingyi Yu, and Shenghua Gao. 2018. Gaze prediction in dynamic 360 immersive videos. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5333--5342.Google ScholarCross Ref
- Praveen Kumar Yadav and Wei Tsang Ooi. 2020. Tile rate allocation for 360-degree tiled adaptive video streaming. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 3724--3733. Google ScholarDigital Library
- Jun Yao, Salil S Kanhere, and Mahbub Hassan. 2008. An empirical study of bandwidth predictability in mobile computing. In Proceedings of the third ACM International Workshop on Wireless Network Testbeds, Experimental Evaluation and Characterization. ACM, 11--18. Google ScholarDigital Library
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 325--338. Google ScholarDigital Library
- Yuanxing Zhang, Pengyu Zhao, Kaigui Bian, Yunxin Liu, Lingyang Song, and Xiaoming Li. 2019. DRL360: 360-degree video streaming with deep reinforcement learning. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 1252--1260.Google ScholarDigital Library
- Chao Zhou, Zhenhua Li, and Yao Liu. 2017. A measurement study of oculus 360 degree video streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 27--37. Google ScholarDigital Library
Index Terms
- TBRA: Tiling and Bitrate Adaptation for Mobile 360-Degree Video Streaming
Recommendations
An HTTP/2-Based Adaptive Streaming Framework for 360° Virtual Reality Videos
MM '17: Proceedings of the 25th ACM international conference on MultimediaVirtual Reality (VR) devices are becoming accessible to a large public, which is going to increase the demand for 360° VR videos. VR videos are often characterized by a poor quality of experience, due to the high bandwidth required to stream the 360° ...
LiveROI: region of interest analysis for viewport prediction in live mobile virtual reality streaming
MMSys '21: Proceedings of the 12th ACM Multimedia Systems ConferenceVirtual reality (VR) streaming can provide immersive video viewing experience to the end users but with huge bandwidth consumption. Recent research has adopted selective streaming to address the bandwidth challenge, which predicts and streams the user's ...
Tiled streaming for layered 3D virtual reality videos with viewport prediction
AbstractIn recent years, the demand of 3D video services has gradually increased. More and more bandwidth hungry applications are proposed, such as immersive media services which need a virtual reality (VR) headset and 3D VR videos to provide users ...
Comments