Skip to main content
SpringerLink
Log in

Offline Downloading: A Comparative Study

  • Chapter
  • First Online:
Content Distribution for Mobile Internet: A Cloud-based Approach

Abstract

Although Internet access has become more ubiquitous in recent years, most users in China still suffer from low-quality connections, especially when downloading large files. To address this issue, hundreds of millions of China’s users have resorted to technologies that allow for “offline downloading,” where a proxy is employed to pre-download the user’s requested file and then deliver the file at her convenience. In this chapter, we examine two typical implementations of offline downloading: the cloud-based approach and the smart AP (access point) based approach. Using a large-scale dataset collected from a major cloud-based system and comprehensive benchmarks of popular smart APs, we find that the two approaches are complementary while also being subject to distinct performance bottlenecks. Driven by these results, we design and implement a proof-of-concept middleware called ODR (Offline Downloading Redirector) to help users get rid of performance bottlenecks. We feel that offline downloading has broad applicability to other areas of the world that lack broadband penetration. By deploying offline downloading technologies, coupled with our proposed ODR middleware, the Internet experiences for users in many parts of the world can be improved.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that the configuration and engineering of Xunlei and Baidu CloudDisk may differ from those of Xuanfeng. Hence, their performance bottlenecks may also be different.

  2. 2.

    Some users of Xuanfeng did not report their access bandwidth. For these users, we use the peak fetching speed recorded in the fetching trace to approximate their access bandwidth.

  3. 3.

    Each selected request record should contain the user’s access bandwidth information so that we can approximate the user’s real network connection characteristics during our benchmarks.

  4. 4.

    ODR maintains a web cookie at the user side (if her web browser permits), so that the user does not need to repeatedly input the auxiliary information every time.

  5. 5.

    The user’s ISP information is obtained based on her IP address with the help of the APNIC service (http://www.apnic.net), a major service provider for IP address collecting/resolving in Asia Pacific.

References

  1. Chetty, M., Sundaresan, S., Muckaden, S., Feamster, N., Calandro, E.: Measuring broadband performance in South Africa. In: Proceedings of the 4th ACM Symposium on Computing for Development (DEV), p. 1 (2013)

    Google Scholar 

  2. Fahad, A., Chen, Z., Shen, K., Bigham, J., Fahad, A.: An evaluation of web acceleration techniques for the developing world. In: Proceedings of the 6th USENIX/ACM Workshop on Networked Systems for Developing Regions (NSDR) (2012)

    Google Scholar 

  3. Grover, S., Park, M.S., Sundaresan, S., Burnett, S., Kim, H., Ravi, B., Feamster, N.: Peeking behind the NAT: an empirical study of home networks. In: Proceedings of the 13th ACM Internet Measurement Conference (IMC), pp. 377–390 (2013)

    Google Scholar 

  4. Johnson, D., Belding, E., Mudenda, C.: Kwaabana: file sharing for rural networks. In: Proceedings of the 4th ACM Symposium on Computing for Development (DEV), p. 4 (2013)

    Google Scholar 

  5. Zaki, Y., Chen, J., Pötsch, T., Ahmad, T., Subramanian, L.: Dissecting web latency in Ghana. In: Proceedings of the 14th ACM Internet Measurement Conference (IMC), pp. 241–248 (2014)

    Google Scholar 

  6. Zheleva, M., Schmitt, P., Vigil, M., Belding, E.: The increased bandwidth fallacy: performance and usage in Rural Zambia. In: Proceedings of the 4th ACM Symposium on Computing for Development (DEV), p. 2 (2013)

    Google Scholar 

  7. FCC raises broadband definition to 25Mbps, Chairman mocks ISPs. http://www.extremetech.com/mobile/198583-fcc-raises-broadband-definition-to-25mbps-chairman-mocks-isps

  8. Key ICT indicators for developed and developing countries and the world (totals and penetration rates). http://www.itu.int/en/ITU-D/Statistics/Documents/statistics/2014/ITU_Key_2005-2014_ICT_data.xls

  9. Bischof, Z.S., Bustamante, F.E., Stanojevic, R.: Need, want, can afford C broadband markets and the behavior of users. In: Proceedings of the 14th ACM Internet Measurement Conference (IMC), pp. 73–86 (2014)

    Google Scholar 

  10. Delay-tolerant networking (DTN) Wiki Page. http://en.wikipedia.org/wiki/Delay-tolerant_networking

  11. Isaacman, S., Martonosi, M.: Low-infrastructure methods to improve internet access for mobile users in emerging regions. In: Proceedings of the 20th International World Wide Web Conference (WWW), pp. 473–482 (2011)

    Google Scholar 

  12. China’s Broadband Penetration Is Increasingly Lagging Behind Developed Nations, Says MIIT’s Research Head. http://techcrunch.com/2013/03/21/china-broadband-laggin

  13. Statistics of China Internet Users. http://www.internetlivestats.com/internet-users/china

  14. The State of Broadband 2014—A Report by the Broadband Commission. http://www.broadbandcommission.org/documents/reports/bb-annualreport2014.pdf

  15. Li, Z., Wilson, C., Xu, T., Liu, Y., Lu, Z., Wang, Y.: Offline downloading in China: a comparative study. In: Proceedings of the 15th ACM Internet Measurement Conference (IMC), pp. 473–486 (2015)

    Google Scholar 

  16. 100,000 MiWiFi smart AP devices are sold out in just 59 seconds. http://bbs.xiaomi.cn/thread-9658495-1-1.html

  17. Baidu CloudDisk offline downloading system. http://pan.baidu.com

  18. HiWiFi smart AP is striving towards 5,000,000 sales. http://www.pcpop.com/doc/1/1002/1002782.shtml

  19. Xuanfeng offline downloading system. http://xf.qq.com

  20. Xunlei offline downloading system. http://lixian.xunlei.com

  21. Ao, N., Xu, Y., Chen, C., Guo, Y.: Offline downloading: a non-traditional cloud-accelerated and peer-assisted content distribution service. In: Proceedings of the IEEE International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), pp. 81–88 (2012)

    Google Scholar 

  22. Huang, Y., Li, Z., Liu, G., Dai, Y.: Cloud download: using cloud utilities to achieve high-quality content distribution for unpopular videos. In: Proceedings of the 19th ACM International Conference on Multimedia (ACM-MM), pp. 213–222 (2011)

    Google Scholar 

  23. Offline Downloading: the Baidu Wikipedia page. http://baike.baidu.com/view/2718066.htm

  24. Offline (movie) Downloading for MiWiFi. http://www.mi.com/miwifi/movie-download

  25. Zhou, Y., Fu, Z., Chiu, D.M., Huang, Y.: An adaptive cloud downloading service. IEEE Trans. Multimed. (TMM) 15(4), 802–810 (2013)

    Article  Google Scholar 

  26. HiWiFi smart AP. http://www.hiwifi.com

  27. MiWiFi smart AP. http://www.miwifi.com

  28. Newifi smart AP. http://www.newifi.com

  29. A Collection of the Best Offline Downloading Tools. http://jingyan.baidu.com/article/636f38bb295e9bd6b84610e9.html

  30. How to Offline Download? Which is the Best Offline Downloading Tool? http://jingyan.baidu.com/article/a65957f4fe63c424e67f9b92.html

  31. An Evaluation Report of HiWiFi. http://news.mydrivers.com/1/279/279305_all.htm

  32. An Evaluation Report of MiWiFi. http://www.geekpark.net/read/view/195133

  33. An Evaluation Report of NewiFi. http://www.itbear.com.cn/n112446c93.aspx

  34. A Comparison of Nine Smart APs. http://net.zol.com.cn/478/4788494.html

  35. HiWiFi vs. Newifi: A Benchmark Test. http://test.smzdm.com/pingce/p/20574

  36. Three Faults of HiWiFi. http://digi.tech.qq.com/a/20131112/002037.htm

  37. Chen, G., Li, Z.: Peer-to-Peer Network: Structure, Application and Design. Tsinghua University Press (2007)

    Google Scholar 

  38. HiWiFi APPs. http://bbs.hiwifi.com/thread-22663-1-1.html

  39. MiWiFi: System Options. http://www.mi.com/miwifi#op

  40. Offline Downloading with Newifi and Xunlei. http://jingyan.baidu.com/article/3d69c5517049e1f0cf02d7a4.html

  41. URL Droplet offline downloading system. http://www.urldroplet.com

  42. Amazon Silk web browser. http://amazonsilk.wordpress.com

  43. Linksys Smart WiFi Router. http://www.linksys.com/en-us/smartwifi

  44. Ahlgren, B., Dannewitz, C., Imbrenda, C., Kutscher, D., Ohlman, B.: A survey of information-centric networking. IEEE Commun. 50(7), 26–36 (2012)

    Article  Google Scholar 

  45. Tian, Y., Dey, R., Liu, Y., Ross, K.: Topology mapping and geolocating for China’s internet. IEEE Trans. Parallel Distrib. Syst. (TPDS) 24(9), 1908–1917 (2013)

    Article  Google Scholar 

  46. Choffnes, D., Bustamante, F.E.: Taming the torrent: a practical approach to reducing cross-ISP traffic in peer-to-peer systems. ACM SIGCOMM Comput. Commun. Rev. (CCR) 38(4), 363–374 (2008)

    Article  Google Scholar 

  47. Xie, H., Yang, R., Krishnamurthy, A., Liu, Y.G., Silberschatz, A.: P4P: provider portal for applications. ACM SIGCOMM Comput. Commun. Rev. (CCR) 38(4), 351–362 (2008)

    Article  Google Scholar 

  48. China-Unicom ISP. http://www.chinaunicom.com.cn

  49. China-Telecom ISP. http://www.chinatelecom.com.cn

  50. China-Mobile ISP. http://www.10086.cn

  51. CERNET (China Education and Research Network) ISP. http://www.cernet.edu.cn

  52. HiWiFi Introduction and History. http://www.hiwifi.com/about

  53. OpenWrt operating system. http://openwrt.org

  54. Opkg (Open PacKaGe Management) web site. http://code.google.com/p/opkg

  55. aria2: The next generation download utility. http://aria2.sourceforge.net

  56. Breslau, L., Cao, P., Fan, L., Phillips, G., Shenker, S.: Web caching and zipf-like distributions: evidence and implications. In: Proceedings of the 18th IEEE International Conference on Computer Communications (INFOCOM), pp. 126–134 (1999)

    Google Scholar 

  57. Guo, L., Tan, E., Chen, S., Xiao, Z., Zhang, X.: The stretched exponential distribution of internet media access patterns. In: Proceedings of the 27th ACM Symposium on Principles of Distributed Computing (PODC), pp. 283–294 (2008)

    Google Scholar 

  58. Gummadi, K.P., Dunn, R.J., Saroiu, S., Gribble, S.D., Levy, H.M., Zahorjan, J.: Measurement, modeling, and analysis of a peer-to-peer file-sharing workload. ACM SIGOPS Oper. Syst. Rev. (OSR) 37(5), 314–329 (2003)

    Article  Google Scholar 

  59. Ganjam, A., Jiang, J., Liu, X., Sekar, V., Siddiqi, F., Stoica, I., Zhan, J., Zhang, H.: C3: Internet-scale control plane for video quality optimization. In: Proceedings of the 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI), pp. 131–144 (2015)

    Google Scholar 

  60. Li, Z., Cao, J., Chen, G.: Continustreaming: achieving high playback continuity of gossip-based peer-to-peer streaming. In: Proceedings of the 22nd IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 1–12 (2008)

    Google Scholar 

  61. Cohen, B.: Incentives build robustness in bittorrent. In: Proceedings of the 1st Workshop on Economics of Peer-to-Peer Systems, pp. 68–72 (2003)

    Google Scholar 

  62. Huang, T.Y., Handigol, N., Heller, B., McKeown, N., Johari, R.: Confused, timid, and unstable: picking a video streaming rate is hard. In: Proceedings of the 12th ACM Internet Measurement Conference (IMC), pp. 225–238 (2012)

    Google Scholar 

  63. Krishnan, S., Sitaraman, R.: Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs. IEEE/ACM Trans. Netw. (TON) 21(6), 2001–2014 (2013)

    Article  Google Scholar 

  64. Li, Z., Huang, Y., Liu, G., Wang, F., Liu, Y., Zhang, Z.L., Dai, Y.: Challenges, designs, and performances of large-scale open-P2SP content distribution. IEEE Trans. Parallel Distrib. Syst. (TPDS) 24(11), 2181–2191 (2013)

    Article  Google Scholar 

  65. Li, Z., Zhang, T., Huang, Y., Zhang, Z.L., Dai, Y.: Maximizing the bandwidth multiplier effect for hybrid cloud-P2P content distribution. In: Proceedings of the 20th IEEE/ACM International Workshop on Quality of Service (IWQoS), pp. 1–9 (2012)

    Google Scholar 

  66. Sundaresan, S., De Donato, W., Feamster, N., Teixeira, R., Crawford, S., Pescapè, A.: Broadband internet performance: a view from the gateway. ACM SIGCOMM Comput. Commun. Rev. (CCR) 41(4), 134–145 (2011)

    Article  Google Scholar 

  67. Tencent PC Manager. http://guanjia.qq.com

  68. Baidu Guard. http://anquan.baidu.com/weishi

  69. 360 Security Guard. http://www.360.cn/weishi

  70. Huang, T.Y., Johari, R., McKeown, N., Trunnell, M., Watson, M.: A buffer-based approach to rate adaptation: evidence from a large video streaming service. In: Proceedings of the 2014 ACM Conference on Communication Architectures, Protocols and Applications (SIGCOMM), pp. 187–198 (2014)

    Google Scholar 

  71. Finamore, A., Mellia, M., Gilani, Z., Papagiannaki, K., Erramilli, V., Grunenberger, Y.: Is there a case for mobile phone content pre-staging? In: Proceedings of the 9th ACM Conference on Emerging Networking EXperiments and Technologies (CoNEXT), pp. 321–326 (2013)

    Google Scholar 

  72. Low Extra Delay Background Transport (LEDBAT), IETF RFC 6817. http://datatracker.ietf.org/doc/rfc6817

  73. Li, Z., Huang, Y., Liu, G., Wang, F., Zhang, Z.L., Dai, Y.: Cloud transcoder: bridging the format and resolution gap between internet videos and mobile devices. In: Proceedings of the 22nd SIGMM Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV), pp. 33–38 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tsinghua University, Beijing, China

    Zhenhua Li & Yunhao Liu

  2. Peking University, Beijing, China

    Yafei Dai

  3. Shanghai Jiao Tong University, Shanghai, China

    Guihai Chen

Authors
  1. Zhenhua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yafei Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guihai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunhao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenhua Li .

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media Singapore

About this chapter

Cite this chapter

Li, Z., Dai, Y., Chen, G., Liu, Y. (2016). Offline Downloading: A Comparative Study. In: Content Distribution for Mobile Internet: A Cloud-based Approach. Springer, Singapore. https://doi.org/10.1007/978-981-10-1463-5_5

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-1463-5_5

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-1462-8

  • Online ISBN: 978-981-10-1463-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation