Skip to main content
SpringerLink
Log in

A simple model to exploit reliable algorithms in cloud federations

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Exploiting resources belonging to multiple cloud providers in an efficient way is still an open issue for distributed computing. Scheduling algorithms based on heuristic, probabilistic, queue theory, or complex soft computing methods are suitable to tackle the heterogeneity and dynamism present in cloud federations. Nevertheless, the available brokering tools are focused on the deployment of services on-demand. The systems able to accomplish high-throughput calculations, such as the pilot-job systems, do not support the inclusion of these algorithms due to their lack of adaptability. The recent implementation of cloud drivers for the GWpilot framework allows developers to profit from its flexibility, compatibility and scheduling features. Moreover, the framework allows the personalised characterisation of cloud resources that those algorithms require, overcoming their lack of trustworthiness in the information provided by the cloud services. In this work, a simple model together with a methodology to couple scheduling software with GWpilot is presented. To demonstrate the suitability of the approach, a legacy self-scheduler specialised on reliable executions in dynamic environments has been stacked and tested on the EGI FedCloud infrastructure with the Nagano legacy application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://www.egi.eu/infrastructure/cloud/.

  2. http://aws.amazon.com.

  3. https://appdb.egi.eu/store/vo/image/de355bfb-5781-5b0c-9ccd-9bd3d0d2be06.

References

  • Abdullah M, Othman M (2013) Cost-based multi-QoS job scheduling using divisible load theory in cloud computing. In: International Conference on Computational Science (ICCS 2013), Elsevier, Barcelona, Spain, Procedia Computer Science, vol 18, pp 928–935. doi:10.1016/j.procs.2013.05.258

  • Aceto G, Botta A, de Donato W, Pescapè A (2013) Cloud monitoring: a survey. Comp Netw 57(9):2093–2115. doi:10.1016/j.comnet.2013.04.001

    Article  Google Scholar 

  • Anastasi GF, Carlini E, Coppola M, Dazzi P (2014) BROKAGE: a genetic approach for QoS cloud brokering. In: 7th IEEE International Conference on Cloud Computing (IEEE CLOUD 2014), Alaska. USA, pp 304–311. doi:10.1109/CLOUD.2014.49

  • Andreozzi S, Burke S, Ehm F, Field L, Galang G, Konya B, Litmaath M, Millar P, Navarro JP (2009) GLUE Specification v. 2.0. http://www.ogf.org/documents/GFD.147, GFD 147

  • Babu D, Venkata P (2013) Honey bee behavior inspired load balancing of tasks in cloud computing environments. Appl Soft Comp 13(5):2292–2303. doi:10.1016/j.asoc.2013.01.025

    Article  Google Scholar 

  • Bala A, Chana I (2015) Autonomic fault tolerant scheduling approach for scientific workflows in Cloud computing. Concur Eng 23(1):27–39. doi:10.1177/1063293X14567783

    Article  Google Scholar 

  • Camarasu-Pop S, Glatard T, da Silva RF, Gueth P, Sarrut D, Benoit-Cattin H (2013) Monte Carlo simulation on heterogeneous distributed systems: a computing framework with parallel merging and checkpointing strategies. Future Gener Comp Syst 29(3):728–738. doi:10.1016/j.future.2012.09.003

    Article  Google Scholar 

  • Chiu CF, Hsu S, Jan SR, Chen JA (2014) Task scheduling based on load approximation in cloud computing environment. In: Future Information Technology, Lecture Notes in Electrical Engineering, vol 309, Springer, Berlin Heidelberg, pp 803–808. doi: 10.1007/978-3-642-55038-6_122

  • Ciuffoletti A (2014) A simple and generic interface for a cloud monitoring service. In: CLOSER 2014 Proceedings of the 4th International Conference on Cloud Computing and Services Science. SCITEPRESS—Science and Technology Publications, Barcelona, Spain, pp 143–150

  • Curnow HJ, Wichmann BA (1976) A synthetic benchmark. Comp J 19(1):43–49. doi:10.1093/comjnl/19.1.43

    Article  Google Scholar 

  • Díaz J, Reyes S, no AN, noz Caro CM, (2009) Derivation of self-scheduling algorithms for heterogeneous distributed computer systems: Application to internet-based grids of computers. Future Gener Comp Syst 25(6):617–626. doi:10.1016/j.future.2008.12.003

  • Foster I, Zhao Y, I Raicu I, Lu S (2008) Cloud computing and grid computing 360-degree compared. In: Grid Computing Environments Workshop (GCE ’08), IEEE, Austin, TX, USA, pp 1–10. doi:10.1109/GCE.2008.4738445

  • Garey M, Johnson D (1979) Computers and intractibility: a guide to the theory of NP-completeness. W. H. Freeman and Co, New York

    MATH  Google Scholar 

  • Glatard T, Camarasu-Pop S (2011) A model of pilot-job resource provisioning on production grids. Parallel Comp 37(10–11):684–692. doi:10.1016/j.parco.2011.04.001

    Article  Google Scholar 

  • Gómez-Iglesias A, Vega-Rodríguez MA, Castejón F, Morales-Ramos E, Cárdenas-Montes M, Reynolds JM (2010) Grid-based metaheuristics to improve a nuclear fusion device. Concurr Computat Pract Exper 22(11):1476–1493. doi:10.1002/cpe.1497

    Google Scholar 

  • Graciani R, Casajús A, Carmona A, Fifield T, Sevior M (2011) Belle-DIRAC setup for using amazon elastic compute cloud. J Grid Comp 9(1):65–79. doi:10.1007/s10723-010-9175-7

    Article  Google Scholar 

  • Grozev N, Buyya R (2014) Inter-Cloud architectures and application brokering: taxonomy and survey. Softw Pract Exper 44, pp. 369–390. doi:10.1002/spe.2168

  • Herrera J (2009) Programming Model for Grid Computing Infrastructures. (in Spanish). PhD thesis, Universidad Complutense de Madrid, Madrid, Spain

  • Huedo E, Montero RS, Llorente IM (2007) A modular meta-scheduling architecture for interfacing with pre-WS and WS Grid resource management services. Future Gener Comp Syst 23(2):252–261. doi:10.1016/j.future.2006.07.013

    Article  Google Scholar 

  • Korkhov VV, Mościcki JT, Krzhizhanovskaya VV (2009) Dynamic workload balancing of parallel applications with user-level scheduling on the grid. Future Gener Comp Syst 25(1):28–34. doi:10.1016/j.future.2008.07.001

    Article  Google Scholar 

  • Kovács J, Marosi AC, Visegrádi A, Farkas Z, Kacsuk P, Lovas R (2015) Boosting gLite with cloud augmented volunteer computing. Future Gener Comp Syst 43–44:12–23. doi:10.1016/j.future.2014.10.005

    Article  Google Scholar 

  • Lu K, Yahyapour R, Wieder P, Yaqub E, Abdullah M, Schloer B, Kotsokalis C (2016) Fault-tolerant service level agreement lifecycle management in clouds using actor system. Future Gener Comp Syst 54:247–259. doi:10.1016/j.future.2015.03.016

    Article  Google Scholar 

  • Lucas-Simarro JL, Moreno-Vozmediano R, Montero RS, Llorente IM (2015) Cost optimization of virtual infrastructures in dynamic multi-cloud scenarios. Concurr Computat Pract Exper 27(9):2260–2277. doi:10.1002/cpe.2972

    Article  Google Scholar 

  • Luckow A, Santcroos M, Zebrowski A, Jha S (2015) Pilot-Data: an abstraction for distributed data. J Parallel Distrib Comp 7980:16–30. doi:10.1016/j.jpdc.2014.09.009

    Article  Google Scholar 

  • Luckow A, Santcroos M, Merzky A, Weidner O, Mantha P, Jha S (2012) P*: A model of pilot-abstractions. In: 8th IEEE International Conference on E-Science (e-Science 2012), Chicago, USA, pp 1–10, 2012. doi:10.1109/eScience.2012.6404423

  • Mhashilkar P, Tiradani A, Holzman B, Larson K, Sfiligoi I, Rynge M (2014) Cloud bursting with GlideinWMS: means to satisfy ever increasing computing needs for scientific workflows. In: 20th International Conference on Computing in High Energy and Nuclear Physics (CHEP2013), IOP Publishing, Journal of Physics: Conference Series, vol 513, p 032069. doi: 10.1088/1742-6596/513/3/032069

  • Mohamed M, Amziani M, Belaïd D, Tata S, Melliti T (2015) An autonomic approach to manage elasticity of business processes in the cloud. Future Gener Comp Syst 50:49–61. doi:10.1016/j.future.2014.10.017

    Article  Google Scholar 

  • Montero R, Huedo E, Llorente I (2006) Benchmarking of high throughput computing applications on grids. Parallel Comp 32(4):267–279. doi:10.1016/j.parco.2005.12.001

    Article  Google Scholar 

  • Moon YH, Youn CH (2015) Multihybrid job scheduling for fault-tolerant distributed computing in policy-constrained resource networks. Comp Netw 82:81–95. doi:10.1016/j.comnet.2015.02.030

    Article  Google Scholar 

  • Moreno-Vozmediano R, Montero RS, Llorente IM (2012) IaaS cloud architecture: from virtualized datacenters to federated cloud infrastructures. Computer 45(12):65–72. doi:10.1109/MC.2012.76

    Article  Google Scholar 

  • Mościcki JT (2011) Understanding and Mastering Dynamics in Computing Grids: Processing Moldable Tasks with User-Level Overlay. PhD thesis, Universiteit van Amsterdam, Nederlands

  • Mościcki JT, Lamannaa M, Bubak M, Sloot PMA (2011) Processing moldable tasks on the grid: late job binding with lightweight user-level overlay. Future Gener Comp Syst 27(6):725–736. doi:10.1016/j.future.2011.02.002

    Article  Google Scholar 

  • Nagano M, Kobayakawa K, Sakaki N, Ando K (2003) Photon yields from nitrogen gas and dry air excited by electrons. Astropart Phys 20(3):293–309. doi:10.1016/S0927-6505(03)00192-0

    Article  Google Scholar 

  • Nagano M, Kobayakawa K, Sakaki N, Ando K (2004) New measurement on photon yields from air and the application to the energy estimation of primary cosmic rays. Astropart Phys 22(3–4):235–248. doi:10.1016/j.astropartphys.2004.08.002

    Article  Google Scholar 

  • Nesmachnow S, Cancela H, Alba E (2010) Heterogeneous computing scheduling with evolutionary algorithms. Soft Comp 15(4):685–701. doi:10.1007/s00500-010-0594-y

    Article  Google Scholar 

  • Panda SK, Gupta I, Jana PK (2015) Allocation-aware Task Scheduling for Heterogeneous Multi-cloud Systems. In: 2nd International Symposium on Big Data and Cloud Computing Challenges (ISBCC ’15), Chennai, India, Procedia Computer Science, vol 50, pp 176–184. doi: 10.1016/j.procs.2015.04.081

  • Parák B, Šustr Z, Feldhaus F, Kasprzakc P, Srbac M (2014) The rOCCI Project: Providing Cloud Interoperability with OCCI 1.1. In: International Symposium on Grids and Clouds (ISGC), Taipei, Taiwan, SISA PoS, pp 1–15

  • Pinedo M (2005) Planning and scheduling in manufacturing and services. Springer Series in Operations Research, Springer, New York, doi:10.1007/b139030

  • Rodríguez-Pascual M, Mayo-García R, Llorente IM (2013) Montera: a framework for efficient execution of Monte Carlo codes on grid infrastructures. Comput Inform 32(1):113–144

    Google Scholar 

  • Rubio-Montero AJ, Huedo E, Mayo-García R (2015c) User-guided provisioning in federated clouds for distributed calculations. In: Workshop on Adaptive Resource Management and Scheduling for Cloud Computing (ARMS-CC 2015), San Sebastián, Spain, Lecture Notes in Computer Science, vol. 9438, pp 60–77. doi: 10.1007/978-3-319-28448-4_5

  • Rubio-Montero AJ, Rodríguez-Pascual MA, Mayo-García R (2015d) Evaluation of an adaptive framework for resilient Monte Carlo executions. In: 30th ACM/SIGAPP Symposium On Applied Computing (SAC’15), Salamanca, Spain, pp 448–455. doi:10.1145/2695664.2695890

  • Rubio-Montero AJ, Castejón F, Huedo E, Mayo-García R (2015a) A novel pilot job approach for improving the execution of distributed codes: application to the study of ordering in collisional transport in fusion plasmas. Concurr Computat Pract Exper 27(13):3220–3244. doi:10.1002/cpe.3301

    Article  Google Scholar 

  • Rubio-Montero AJ, Huedo E, Castejón F, Mayo-García R (2015b) GWpilot: enabling multi-level scheduling in distributed infrastructures with GridWay and pilot jobs. Future Gener Comp Syst 45:25–52. doi:10.1016/j.future.2014.10.003

    Article  Google Scholar 

  • Sajid M, Razaa Z (2015) Turnaround time minimization-based static scheduling model using task duplication for fine-grained parallel applications onto hybrid cloud environment. IETE J Res. doi:10.1080/03772063.2015.1075911 (In press)

  • Saleh A (2013) An efficient grid-scheduling strategy based on a fuzzy matchmaking approach. Soft Comp 17(3):467–487. doi:10.1007/s00500-012-0920-7

    Article  MathSciNet  Google Scholar 

  • Sheikhalishahi M, Wallace R, Grandinetti L, Vázquez-Poletti JL, Guerriero F (2015) A multi-dimensional job scheduling. Future Gener Comp Syst. doi:10.1016/j.future.2015.03.014

  • Shie MR, Liu CY, Lee YF, Lin YC, Lai KC (2014) Distributed scheduling approach based on game theory in the federated cloud. In: International Conference on Information Science and Applications (ICISA 2014), IEEE CS Press, Seoul, South Corea, pp 1–4. doi:10.1109/ICISA.2014.6847388

  • Smanchat S, Viriyapant K (2015) Taxonomies of workflow scheduling problem and techniques in the cloud. Future Gener Comp Syst 52:1–12. doi:10.1016/j.future.2015.04.019

    Article  Google Scholar 

  • Snyder B, Ringenberg J, Green R, Devabhaktuni V, Alam M (2015) Evaluation and design of highly reliable and highly utilized cloud computing systems. J Cloud Comput Adv Syst Appl 4(11):1–16. doi:10.1186/s13677-015-0036-6

    Google Scholar 

  • Tao F, Feng Y, Zhang L, Liao T (2014) Clps-ga: a case library and pareto solution-based hybrid genetic algorithm for energy-aware cloud service scheduling. Appl Soft Comput 19:264–279. doi:10.1016/j.asoc.2014.01.036

    Article  Google Scholar 

  • Tomás L, Caminero AC, Rana O, Carrión C, Caminero B (2012) A GridWay-based autonomic network-aware metascheduler. Future Gener Comp Syst 28(7):1058–1069. doi:10.1016/j.future.2011.08.019

  • Tordsson J, Montero RS, Moreno-Vozmediano R, Llorente IM (2012) Cloud brokering mechanisms for optimized placement of virtual machines across multiple providers. Future Gener Comp Syst 28(2):358–367. doi:10.1016/j.future.2011.07.003

    Article  Google Scholar 

  • Tzen TH, Ni LM (1993) Trapezoid self-scheduling: a practical scheduling scheme for parallel compilers. IEEE Trans Parallel Distrib Syst 4(1). doi:10.1109/71.205655

  • Vélez JR (2011) Analysis of the air fluorescence induced by electrons for application to cosmic ray detection. PhD thesis, Universidad Complutense de Madrid, Madrid, Spain

  • Wang X, Wang Y, Cui Y (2016) An energy-aware bi-level optimization model for multi-job scheduling problems under cloud computing. Soft Comput 20(1):303–317. doi:10.1007/s00500-014-1506-3

    Article  Google Scholar 

  • Xu B, Peng Z, Xiao F, Gates A, Yu JP (2015) Dynamic deployment of virtual machines in cloud computing using multi-objective optimization. Soft Comput 19(8):2265–2273. doi:10.1007/s00500-014-1406-6

    Article  Google Scholar 

  • Yangui S, Marshall IJ, Laisne JP, Tata S (2014) CompatibleOne: the open source cloud broker. J Grid Comput 12(1):93–109. doi:10.1007/s10723-013-9285-0

    Article  Google Scholar 

  • Zanikolas S, Sakellariou R (2005) A taxonomy of grid monitoring systems. Future Gener Compr Syst 21(1):163–188. doi:10.1016/j.future.2004.07.002

    Article  Google Scholar 

  • Zhani M, Boutaba R (2015) Survivability and fault tolerance in the cloud. John Wiley & Sons Inc, pp 295–308. doi:10.1002/9781119042655.ch12

Download references

Author information

Authors and Affiliations

  1. CIEMAT, Av. Complutense 40, 28040, Madrid, Spain

    A. J. Rubio-Montero, M. A. Rodríguez-Pascual & R. Mayo-García

Authors
  1. A. J. Rubio-Montero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. A. Rodríguez-Pascual
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Mayo-García
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. J. Rubio-Montero.

Ethics declarations

Funding

This work was supported by the COST Actions BETTY (IC 1201) and NESUS (IC1305) and partially funded by the Spanish Ministry of Economy and Competitiveness project CODEC2 (TIN2015-63562-R).

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by F. Pop, C. Dobre and A. Costan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rubio-Montero, A.J., Rodríguez-Pascual, M.A. & Mayo-García, R. A simple model to exploit reliable algorithms in cloud federations . Soft Comput 21, 4543–4555 (2017). https://doi.org/10.1007/s00500-016-2143-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2143-9

Keywords

Search

Navigation