Ang Li
ABSTRACT
While many public cloud providers offer pay-as-you-go computing, their varying approaches to infrastructure, virtualization, and software services lead to a problem of plenty. To help customers pick a cloud that fits their needs, we develop CloudCmp, a systematic comparator of the performance and cost of cloud providers. CloudCmp measures the elastic computing, persistent storage, and networking services offered by a cloud along metrics that directly reflect their impact on the performance of customer applications. CloudCmp strives to ensure fairness, representativeness, and compliance of these measurements while limiting measurement cost. Applying CloudCmp to four cloud providers that together account for most of the cloud customers today, we find that their offered services vary widely in performance and costs, underscoring the need for thoughtful provider selection. From case studies on three representative cloud applications, we show that CloudCmp can guide customers in selecting the best-performing provider for their applications.
- Amazon SimpleDB. http://aws.amazon.com/simpledb/.Google Scholar
- CloudCmp Project Website. http://cloudcmp.net.Google Scholar
- CloudStatus. http://www.cloudstatus.com.Google Scholar
- Comparing Amazon EC2 performance with other cloud/VPS hosting options. and real hardware. http://www.paessler.com/blog/2009/04/14/prtg-7/comparing-amazon-ec2-performance-with-other-cloudvps-hosting-options-and-real-hardware.Google Scholar
- Iperf. http://iperf.sourceforge.net.Google Scholar
- Java TPC-W Implementation. http://www.ece.wisc.edu/~pharm/tpcw.shtml.Google Scholar
- PlanetLab. http://www.planet-lab.org.Google Scholar
- Rackspace Cloud Servers versus Amazon EC2: Performance Analysis. http://www.thebitsource.com/featured-posts/rackspace-cloud-servers-versus-amazon-ec2-performance-analysis/.Google Scholar
- SPEC CPU2006 Benchmark. http://www.spec.org/cpu2006.Google Scholar
- SPEC Java Virtual Machine Benchmark 2008. http://www.spec.org/jvm2008/.Google Scholar
- Standard Performance Evaluation Corporation. http://www.spec.org.Google Scholar
- TPC Benchmark W. http://www.tpcw.org/tpcw.Google Scholar
- M. Armbrust, A. Fox, R. Griffith, A.D. Joseph, R.H. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and Others. Above the clouds: A berkeley view of cloud computing. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-28, 2009.Google Scholar
- B.F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. Benchmarking Cloud Serving Systems with YCSB. In ACM Symposium on Cloud Computing, 2010. Google ScholarDigital Library
- Simson Garfinkel. An Evaluation of Amazon s Grid Computing Services : EC2, S3 and SQS. Harvard University, Tech. Rep. TR-08-07.Google Scholar
- Seth Gilbert and Nancy Lynch. Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. In ACM SIGACT News, 2002. Google ScholarDigital Library
- Andreas Haeberlen. A Case for the Accountable Cloud. In ACM SIGOPS LADIS, 2009.Google Scholar
- Mohammad Hajjat, Xin Sun, Yu-Wei Eric Sung, David Maltz, Sanjay Rao, Kunwadee Sripanidkulchai, and Mohit Tawarmalani. Cloudward Bound: Planning for Beneficial Migration of Enterprise Applications to the Cloud. In ACM SIGCOMM, 2010. Google ScholarDigital Library
- Ron Kohavi, Randal M. Henne, and Dan Sommerfield. Practical guide to controlled experiments on the web. In ACM KDD, volume 2007. ACM Press, 2007. Google ScholarDigital Library
- Ang Li, Xiaowei Yang, Srikanth Kandula, and Ming Zhang. CloudCmp: Shopping for a Cloud Made Easy. In USENIX HotCloud, 2010. Google ScholarDigital Library
- Jitendra Padhye, Victor Firoiu, Don Towsley, and Jim Kurose. Modeling TCP Throughput: A Simple Model and its Empirical Validation. In ACM SIGCOMM, 1998. Google ScholarDigital Library
- Himanshu Raj, R Nathuji, A Singh, and Paul England. Resource management for isolation enhanced cloud services. In ACM Workshop on Cloud Computing Security, pages 77--84, 2009. Google ScholarDigital Library
- Thomas Ristenpart, Eran Tromer, Hovav Shacham, and Stefan Savage. Hey, You, Get Off of My Cloud: Exploring Information Leakage in Third-Party Compute Clouds. In ACM CCS, Chicago, 2009. Google ScholarDigital Library
- Nuno Santos, K.P. Gummadi, and Rodrigo Rodrigues. Towards trusted cloud computing. In HotCloud, 2009. Google ScholarDigital Library
- E. Walker. Benchmarking amazon EC2 for high-performance scientific computing. USENIX Login, 2008.Google Scholar
- Guohui Wang and T. S. Eugene Ng. The Impact of Virtualization on Network Performance of Amazon EC2 Data Center. In IEEE INFOCOM, 2010. Google ScholarDigital Library
- Jonathan S. Ward. A Performance Comparison of Clouds: Amazon EC2 and Ubuntu Enterprise Cloud. SICSA DemoFEST, 2009.Google Scholar
- T. Wood, A. Gerber, KK Ramakrishnan, P. Shenoy, and J. Van Der Merwe. The Case for Enterprise-Ready Virtual Private Clouds. In USENIX HotCloud, 2009. Google ScholarDigital Library
Index Terms
- CloudCmp: comparing public cloud providers
Recommendations
CloudCmp: shopping for a cloud made easy
HotCloud'10: Proceedings of the 2nd USENIX conference on Hot topics in cloud computingCloud computing has gained much popularity recently, and many companies now offer a variety of public cloud computing services, such as Google Ap-pEngine, Amazon AWS, and Microsoft Azure. These services differ in service models and pricing schemes, ...
Minimizing Cost and Maximizing Performance for Cloud Platforms
Middleware'20 Doctoral Symposium: Proceedings of the 21st International Middleware Conference Doctoral SymposiumWe are witnessing the rapid growth of cloud computing with the proliferation of tenants adopting cloud for elasticity, availability, and flexibility for a plethora of applications. To efficiently cater for different tenant requirements, cloud providers ...
Glaucus: predicting computing-intensive program's performance for cloud customers
ICIC'13: Proceedings of the 9th international conference on Intelligent Computing TheoriesAs Cloud computing has gained much popularity recently, many organizations consider transmitting their large-scale computing-intensive programs to cloud. However, cloud service market is still in its infant stage. Many companies offer a variety of cloud ...
Comments