Abstract
Benchmarking the containerized web-applications across multiple cloud gives web-application owners more chance to deploy their applications on cheaper host while meeting their performance requirements. However, benchmarking a large number of cloud hosts (about 267 cloud providers in the world) to find a flexible deployment option becomes a grand challenge. Users need to evaluate as many hosts as possible to find an option which offers expected performance at the lowest price. It is also necessary to benchmark the hosts for longer duration so that it can capture the uncertainty of cloud environment.
In this paper, we present Smart Docker Benchmarking Orchestrator (SDBO), a general orchestrator that automatically benchmarks containerized web-applications in multi-cloud environment. At the same time, SDBO is able to maximize the numbers of evaluated cloud providers and type of hosts without exceeding users’ budgets. Moreover, we propose a flexible execution module which enhances SDBO ’s ability to capture the performance variation of benchmark web-application for longer period of time in the defined users’ budgets.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Docker stats. https://docs.docker.com/engine/reference/commandline/stats/. Accessed 17 Apr 2019
I/o characteristics and monitoring. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-io-characteristics.html/. Accessed 17 Apr 2019
Chhetri, M.B., et al.: Smart cloudbench—a framework for evaluating cloud infrastructure performance. Inf. Syst. Front. 18(3), 413–428 (2016)
Duan, Q.: Cloud service performance evaluation: status, challenges, and opportunities - a survey from the system modeling perspective. Digit. Commun. Networks 3(2), 101–111 (2017)
Jha, D.N., Garg, S., Jayaraman, P.P., Buyya, R., Li, Z., Morgan, G., Ranjan, R.: A study on the evaluation of HPC microservices in containerized environment. Concurrency Comput. Pract. Experience e5323 (2019)
Jha, D.N., Nee, M., Wen, Z., Zomaya, A., Ranjan, R.: SmartDBO: smart docker benchmarking orchestrator for web-application. In: The World Wide Web Conference, pp. 3555–3559. ACM (2019)
Kozhirbayev, Z., Sinnott, R.O.: A performance comparison of container-based technologies for the cloud. Future Gener. Comput. Syst. 68, 175–182 (2017)
Leitner, P., Cito, J.: Patterns in the chaos - a study of performance variation and predictability in public iaas clouds. ACM Trans. Internet Technol. 16(3), 15:1–15:23 (2016)
Ranjan, R., Benatallah, B., Dustdar, S., Papazoglou, M.P.: Cloud resource orchestration programming: overview, issues, and directions. IEEE Internet Comput. 19(5), 46–56 (2015)
Scheuner, J., Cito, J., Leitner, P., Gall, H.: Cloud workbench: benchmarking IaaS providers based on infrastructure-as-code. In: Proceedings of the 24th International Conference on World Wide Web, pp. 239–242. ACM (2015)
Serrano, D., et al.: SLA guarantees for cloud services. Future Gener. Comput. Syst. 54, 233–246 (2016)
Sharma, P., Chaufournier, L., Shenoy, P., Tay, Y.C.: Containers and virtual machines at scale: a comparative study. In: Proceedings of the 17th International Middleware Conference, Middleware 2016, pp. 1:1–1:13 (2016)
Silva, M., Hines, M.R., Gallo, D., Liu, Q., Ryu, K.D., Da Silva, D.: CloudBench: experiment automation for cloud environments. In: 2013 IEEE International Conference on Cloud Engineering (IC2E), pp. 302–311. IEEE (2013)
Sobel, W., et al.: Cloudstone: multi-platform, multi-language benchmark and measurement tools for web 2.0. In: Proceedings of CCA (2008)
Varghese, B., Subba, L.T., Thai, L., Barker, A.: Doclite: a docker-based lightweight cloud benchmarking tool. In: 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 213–222. IEEE (2016)
Weerasiri, D., Barukh, M.C., Benatallah, B., Sheng, Q.Z., Ranjan, R.: A taxonomy and survey of cloud resource orchestration techniques. ACM Comput. Surv. (CSUR) 50(2), 26 (2017)
Xavier, M.G., Neves, M.V., De Rose, C.A.F.: A performance comparison of container-based virtualization systems for mapreduce clusters. In: 2014 22nd Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pp. 299–306. IEEE (2014)
Zhang, Q., Liu, L., Pu, C., Dou, Q., Wu, L., Zhou, W.: A comparative study of containers and virtual machines in big data environment. In: 2018 IEEE 11th International Conference on Cloud Computing (CLOUD), pp. 178–185. IEEE (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Jha, D.N., Wen, Z., Li, Y., Nee, M., Koutny, M., Ranjan, R. (2019). A Cost-Efficient Multi-cloud Orchestrator for Benchmarking Containerized Web-Applications. In: Cheng, R., Mamoulis, N., Sun, Y., Huang, X. (eds) Web Information Systems Engineering – WISE 2019. WISE 2020. Lecture Notes in Computer Science(), vol 11881. Springer, Cham. https://doi.org/10.1007/978-3-030-34223-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-34223-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34222-7
Online ISBN: 978-3-030-34223-4
eBook Packages: Computer ScienceComputer Science (R0)