skip to main content
10.1145/3474085.3475590acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

TBRA: Tiling and Bitrate Adaptation for Mobile 360-Degree Video Streaming

Published:17 October 2021Publication History

ABSTRACT

Tile-based approach is widely adopted in adaptive 360\textdegree~video streaming systems. Existing QoE-driven streaming approaches usually obtain the tile selection and adjust the bitrate based on the viewport prediction with a fixed tiling, which fail to consider the unstable prediction performance. However, varying the tiling of the video can produce different number of tiles with different sizes, and thus can have distinct impacts on error tolerance for viewport prediction and on decoding complexity for resource-constrained mobile client. In this work, we introduce adaptive tiling into the conventional bitrate adaptation for mobile 360degree~video streaming. We first analyze the impacts of tilings on tile selection and decoding time, which verify the benefit of tiling adaptation in various practical aspects. We then formulate the QoE optimization problem for adaptive tiling and bitrate streaming and discuss the design details of our adaptation algorithm, which can adapt to the performance of viewport prediction and the decoding capabilities of mobile clients in addition to the conventional influencing factors. Finally, the superiority of our proposed approach compared with the state-of-the-art methods is evaluated through extensive trace-driven simulations.

References

  1. Accessed: 2021. FFmpeg. https://ffmpeg.org/.Google ScholarGoogle Scholar
  2. Accessed: 2021. Kvazaar endocer. https://github.com/ultravideo/kvazaar.Google ScholarGoogle Scholar
  3. Accessed: 2021. MediaCodec API. https://developer.android.com/reference/ android/media/MediaCodec.Google ScholarGoogle Scholar
  4. Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: auto-tuning video ABR algorithms to network conditions. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 44--58. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Ghufran Baig, Jian He, Mubashir Adnan Qureshi, Lili Qiu, Guohai Chen, Peng Chen, and Yinliang Hu. 2019. Jigsaw: Robust live 4k video streaming. In Proceedings of the 25th Annual International Conference on Mobile Computing and Networking. ACM, 1--16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Lovish Chopra, Sarthak Chakraborty, Abhijit Mondal, and Sandip Chakraborty. 2021. PARIMA: Viewport Adaptive 360-Degree Video Streaming. In Proceedings of the Web Conference 2021. 2379--2391. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Erwan J David, Jesús Gutiérrez, Antoine Coutrot, Matthieu Perreira Da Silva, and Patrick Le Callet. 2018. A dataset of head and eye movements for 360textdegree videos. In Proceedings of the 9th ACM Multimedia Systems Conference. ACM, 432--437. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Yu Guan, Chengyuan Zheng, Xinggong Zhang, Zongming Guo, and Junchen Jiang. 2019. Pano: Optimizing 360 video streaming with a better understanding of quality perception. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 394--407. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Jian He, Mubashir Adnan Qureshi, Lili Qiu, Jin Li, Feng Li, and Lei Han. 2018. Rubiks: Practical 360-Degree Streaming for Smartphones. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 482--494. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Yifei Huang, Minjie Cai, Zhenqiang Li, and Yoichi Sato. 2018. Predicting gaze in egocentric video by learning task-dependent attention transition. In Proceedings of the European Conference on Computer Vision (ECCV). 754--769.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Junchen Jiang, Vyas Sekar, and Hui Zhang. 2014. Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. IEEE/ACM Transactions on Networking (ToN), Vol. 22, 1 (2014), 326--340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Nan Jiang, Yao Liu, Tian Guo, Wenyao Xu, Viswanathan Swaminathan, Lisong Xu, and Sheng Wei. 2020. QuRate: power-efficient mobile immersive video streaming. In Proceedings of the 11th ACM Multimedia Systems Conference. ACM, 99--111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Nan Jiang, Viswanathan Swaminathan, and Sheng Wei. 2017. Power Evaluation of 360 VR Video Streaming on Head Mounted Display Devices. In Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, 55--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Wen-Chih Lo, Ching-Ling Fan, Jean Lee, Chun-Ying Huang, Kuan-Ta Chen, and Cheng-Hsin Hsu. 2017. 360 video viewing dataset in head-mounted virtual reality. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 211--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Haoran Lv, Qin Yang, Chenglin Li, Wenrui Dai, Junni Zou, and Hongkai Xiong. 2020. SalGCN: Saliency Prediction for 360-Degree Images Based on Spherical Graph Convolutional Networks. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 682--690. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Saeed Mahmoudpour and Peter Schelkens. 2020. Omnidirectional Video Quality Index Accounting for Judder. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, 1 (2020), 61--75.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 197--210. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Stefano Petrangeli, Viswanathan Swaminathan, Mohammad Hosseini, and Filip De Turck. 2017. An http/2-based adaptive streaming framework for 360 virtual reality videos. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 306--314. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Feng Qian, Bo Han, Qingyang Xiao, and Vijay Gopalakrishnan. 2018. Flare: Practical viewport-adaptive 360-degree video streaming for mobile devices. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. ACM, 99--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Giuseppe Ribezzo, Luca De Cicco, Vittorio Palmisano, and Saverio Mascolo. 2020. TAPAS-360: A Tool for the Design and Experimental Evaluation of 360 Video Streaming Systems. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 4477--4480. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jihoon Ryoo, Kiwon Yun, Dimitris Samaras, Samir R Das, and Gregory Zelinsky. 2016. Design and evaluation of a foveated video streaming service for commodity client devices. In Proceedings of the 7th ACM Multimedia Systems Conference. ACM, 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Shu Shi, Varun Gupta, Michael Hwang, and Rittwik Jana. 2019. Mobile VR on edge cloud: a latency-driven design. In Proceedings of the 10th ACM Multimedia Systems Conference. ACM, 222--231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, and Gordon Wetzstein. 2018. Saliency in VR: How do people explore virtual environments? IEEE Transactions on Visualization and Computer Graphics, Vol. 24, 4 (2018), 1633--1642. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Yi Sun, Xiaoqi Yin, Junchen Jiang, Vyas Sekar, Fuyuan Lin, Nanshu Wang, Tao Liu, and Bruno Sinopoli. 2016. CS2P: Improving video bitrate selection and adaptation with data-driven throughput prediction. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 272--285. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jeroen Van Der Hooft, Stefano Petrangeli, Tim Wauters, Rafael Huysegems, Patrice Rondao Alface, Tom Bostoen, and Filip De Turck. 2016. HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks. IEEE Communications Letters, Vol. 20, 11 (2016), 2177--2180.Google ScholarGoogle ScholarCross RefCross Ref
  26. Shuoqian Wang, Xiaoyang Zhang, Mengbai Xiao, Kenneth Chiu, and Yao Liu. 2020. SphericRTC: A System for Content-Adaptive Real-Time 360-Degree Video Communication. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 3595--3603. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Chenglei Wu, Zhihao Tan, Zhi Wang, and Shiqiang Yang. 2017. A dataset for exploring user behaviors in VR spherical video streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 193--198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Chenglei Wu, Ruixiao Zhang, Zhi Wang, and Lifeng Sun. 2020. A Spherical Convolution Approach for Learning Long Term Viewport Prediction in 360 Immersive Video. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 14003--14040.Google ScholarGoogle ScholarCross RefCross Ref
  29. Mengbai Xiao, Chao Zhou, Yao Liu, and Songqing Chen. 2017. Optile: Toward optimal tiling in 360-degree video streaming. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 708--716. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Lan Xie, Zhimin Xu, Yixuan Ban, Xinggong Zhang, and Zongming Guo. 2017. 360probdash: Improving qoe of 360 video streaming using tile-based http adaptive streaming. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 315--323. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Lan Xie, Xinggong Zhang, and Zongming Guo. 2018. Cls: A cross-user learning based system for improving qoe in 360-degree video adaptive streaming. In Proceedings of the 26th ACM International Conference on Multimedia. ACM, 564--572. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Yanyu Xu, Yanbing Dong, Junru Wu, Zhengzhong Sun, Zhiru Shi, Jingyi Yu, and Shenghua Gao. 2018. Gaze prediction in dynamic 360 immersive videos. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5333--5342.Google ScholarGoogle ScholarCross RefCross Ref
  33. Praveen Kumar Yadav and Wei Tsang Ooi. 2020. Tile rate allocation for 360-degree tiled adaptive video streaming. In Proceedings of the 28th ACM International Conference on Multimedia. ACM, 3724--3733. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Jun Yao, Salil S Kanhere, and Mahbub Hassan. 2008. An empirical study of bandwidth predictability in mobile computing. In Proceedings of the third ACM International Workshop on Wireless Network Testbeds, Experimental Evaluation and Characterization. ACM, 11--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 325--338. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Yuanxing Zhang, Pengyu Zhao, Kaigui Bian, Yunxin Liu, Lingyang Song, and Xiaoming Li. 2019. DRL360: 360-degree video streaming with deep reinforcement learning. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications. IEEE, 1252--1260.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Chao Zhou, Zhenhua Li, and Yao Liu. 2017. A measurement study of oculus 360 degree video streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 27--37. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. TBRA: Tiling and Bitrate Adaptation for Mobile 360-Degree Video Streaming

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          MM '21: Proceedings of the 29th ACM International Conference on Multimedia
          October 2021
          5796 pages
          ISBN:9781450386517
          DOI:10.1145/3474085

          Copyright © 2021 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 17 October 2021

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

          Acceptance Rates

          Overall Acceptance Rate995of4,171submissions,24%

          Upcoming Conference

          MM '24
          MM '24: The 32nd ACM International Conference on Multimedia
          October 28 - November 1, 2024
          Melbourne , VIC , Australia

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader