Abstract
Large scale clusters are now being used in shared, multi-tenant scenarios by heterogeneous applications with completely different requirements. In this scenario, it’s useful to explore the intersection of two complementary goals. On one side, energy efficiency is an important factor to consider in this world with increasing operating costs related to energy consumption. On the other side, heterogeneous applications emphasize the problem of distributing the execution capacity among competitive users in a shared setting. In this paper, we address the combination of these two goals by introducing an incentive mechanism to make users report their actual resource requirements, allowing them to dynamically scale-up or down as necessary. In turn, this information is used by the infrastructure operator to shut down resources without reducing the QoS provided to users and effectively reducing energy costs. We show how our mechanism is able to meet the performance requirements of applications without over-provisioning physical resources, which in turn translates into energy savings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Communications of the ACM 51(1), 107–113 (2008)
Barroso, L.A., Hölzle, U.: The datacenter as a computer: An introduction to the design of warehouse-scale machines. Synthesis Lectures on Computer Architecture 4(1), 1–108 (2009)
Rafique, M.M., Rose, B., Butt, A.R., Nikolopoulos, D.S.: Cellmr: A framework for supporting mapreduce on asymmetric cell-based clusters. In: IEEE International Symposium on Parallel & Distributed Processing, IPDPS, pp. 1–12. IEEE (2009)
He, B., Fang, W., Luo, Q., Govindaraju, N.K., Wang, T.: Mars: A mapreduce framework on graphics processors. In: Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques, pp. 260–269. ACM (2008)
White, T.: Hadoop: The definitive guide. O’Reilly Media (2012)
Kaushik, R.T., Bhandarkar, M.: Greenhdfs: Towards an energy-conserving storage-efficient, hybrid hadoop compute cluster. In: Proceedings of the USENIX Annual Technical Conference (2010)
Leverich, J., Kozyrakis, C.: On the energy (in) efficiency of hadoop clusters. ACM SIGOPS Operating Systems Review 44(1), 61–65 (2010)
Lang, W., Patel, J.M.: Energy management for mapreduce clusters. Proceedings of the VLDB Endowment 3(1-2), 129–139 (2010)
Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., Stoica, I.: Job scheduling for multi-user mapreduce clusters. Technical Report UCB/EECS-2009-55, EECS Department, University of California, Berkeley (April 2009)
Sandholm, T., Lai, K.: Mapreduce optimization using regulated dynamic prioritization. In: 11th International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2009, pp. 299–310. ACM, New York (2009)
Sandholm, T., Lai, K.: Dynamic proportional share scheduling in hadoop. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2010. LNCS, vol. 6253, pp. 110–131. Springer, Heidelberg (2010)
Yom-Tov, E., Aridor, Y.: A self-optimized job scheduler for heterogeneous server clusters. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2007. LNCS, vol. 4942, pp. 169–187. Springer, Heidelberg (2008)
Polo, J., Carrera, D., Becerra, Y., Torres, J., Ayguadé, E., Steinder, M., Whalley, I.: Performance-driven task co-scheduling for mapreduce environments. In: IEEE Network Operations and Management Symposium (NOMS), pp. 373–380. IEEE (2010)
Verma, A., Cherkasova, L., Campbell, R.H.: Aria: automatic resource inference and allocation for mapreduce environments. In: Proceedings of the 8th ACM International Conference on Autonomic Computing, ICAC 2011, pp. 235–244. ACM, New York (2011)
Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems 26(2) (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this paper
Cite this paper
León, X., Navarro, L. (2013). Incentives for Dynamic and Energy-Aware Capacity Allocation for Multi-tenant Clusters. In: Altmann, J., Vanmechelen, K., Rana, O.F. (eds) Economics of Grids, Clouds, Systems, and Services. GECON 2013. Lecture Notes in Computer Science, vol 8193. Springer, Cham. https://doi.org/10.1007/978-3-319-02414-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-02414-1_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-02413-4
Online ISBN: 978-3-319-02414-1
eBook Packages: Computer ScienceComputer Science (R0)