A hybrid policy for fault tolerant load balancing in grid computing environments

https://doi.org/10.1016/j.jnca.2011.09.005Get rights and content

Abstract

Due to the emergence of grid computing over the Internet, there is a need for a hybrid load balancing algorithm which takes into account the various characteristics of the grid computing environment. Hence, this research proposes a fault tolerant hybrid load balancing strategy namely AlgHybrid_LB, which takes into account grid architecture, computer heterogeneity, communication delay, network bandwidth, resource availability, resource unpredictability and job characteristics. AlgHybrid_LB juxtaposes the strong points of neighbor-based and cluster based load balancing algorithms. Our main objective is to arrive at job assignments that could achieve minimum response time and optimal computing node utilization. Major achievements include low complexity of proposed approach and drastic reduction of number of additional communications induced due to load balancing. A simulation of the proposed approach using Grid Simulation Toolkit (GridSim) is conducted. Experimental results show that the proposed algorithm performs very well in a large grid environment.

Introduction

Grid computing has emerged as the next-generation distributed computing methodology that aggregates dispersed heterogeneous resources for solving various kinds of large-scale applications in science and engineering. These resources are from a large number of organizations each with a local administration and scheduling entity, which are connected by Wide Area Networks (WAN). In large-scale Grid environments, the underlying network connecting them is heterogeneous and bandwidth across resources varies from link to link. Not limited to Grid, in many of today's distributed computing environments (DCEs), the computers are linked by a delay- and bandwidth-limited communication medium that inherently inflicts tangible delays on inter-node communications and load exchange (Grosan et al., 2007). Unlike the DCEs, a Grid environment is extremely unpredictable: processor capabilities are different and usually unknown, computers may connect and disconnect at any time, and their speeds may change over time (Zomaya and Teh, 2001).

Grid systems are classified into two categories: compute and data grids. In compute grids the main resource that is being managed by the resource management system is compute cycles (i.e. processors); while in data grids the focus is to manage data distributed over geographical locations. The architecture and the services provided by the resource management system are affected by the type of grid system it is deployed in Salleh and Zomaya (1999).

The resource management system is the central component of the grid system. Its basic responsibilities are to accept requests from users, match user requests to available resources for which the user has permission to use and schedule the matched resources. To be able to fully benefit from such grid systems, resource management and scheduling are key grid services, where issues of task allocation and load balancing represent a common challenge for most grids. In a computational grid, at a given time, the task is to allocate the user defined jobs efficiently both by meeting the deadlines and making use of all the available resources (Salleh and Zomaya, 1999).

Computing usage in different science domains allows scientists to use various job-scheduling algorithms to utilize resources effectively. This raises performance in enhancing execution performance and maintaining system load balancing. Still, resources are contributed by idle computers in the grid computing environment. The contributed resources will change with time. This happens when new resources join or old resources exit the grid environment. In this dynamic and time-variant environment, designing a job scheduling algorithm to dynamically change according to the variation in resources requires considering numerous and complicated factors. This complex scheduling will cause greater load on the system, which seriously affects system performance. For this reason, a proper job scheduling algorithm is very important in the grid environment.

Due to uneven job arrival patterns and unequal computing capabilities, some nodes may be overloaded while others may be under-utilized. Load balancing mechanism aims to equally spread the load on each node of the grid, optimizing their utilization, throughput and response (Tang and Chanson, 2000). In order to achieve these goals, the load balancing mechanism should be “fair” in distributing the load across the nodes; by being “fair” we mean that the difference between the “heaviest-loaded” node and the “lightest-loaded” node should be minimized (Li, 2008).

In general, load balancing algorithms can be roughly categorized as centralized, decentralized or hierarchical in terms of location where the load balancing decisions are made (Kandagatla, 2003). In centralized approach, one resource in a distributed system acts as the central controller. It has a global view of the load information in the system and decides how to allocate jobs to other resources. When the system size increases, the global knowledge of the system attributes is prohibitive due to the communication produced overhead and the central controller becomes a system bottleneck and single point of failure. In decentralized approach, all resources in the distributed system are involved in making load balancing decisions (Cao et al., 2005). Since load balancing decisions are distributed, it is costly to let each resource obtain the dynamic state information of the whole system. Hence most algorithms only use partial information stored in the local resource to make a sub-optimal decision. In a hierarchical model, the schedulers are organized in a hierarchy. High level resource entities are scheduled at higher levels and lower level smaller sub-entities are scheduled at lower levels of the scheduler hierarchy. This model is a combination of centralized approach and decentralized approach (Shivaratri et al., 1992).

Load balancing algorithms can be further categorized as static, dynamic or adaptive algorithms based on the type of information on which the load balancing decisions are made (Feng et al., 2000, Willebeek-LeMair and Reeves, 1993). Static algorithms assume that all information including the characteristics of jobs, the resources and communication network are known in advance. This approach is simple and has minimal runtime overhead. However, it has two major disadvantages. Firstly, the workload distribution of many applications cannot be predicted before program execution. Secondly, it assumes that all information remain constant, which may not apply to a grid environment. In contrast to its counterpart, dynamic load balancing algorithms can yield an immediate and direct decision for a newly arriving job without a priori knowledge and hence hardly wastes any processing time. However, this comes at the additional cost of collecting and maintaining load information, hence it is important to keep these overheads within reasonable limits. Adaptive load balancing algorithms are a special class of dynamic algorithms. They adapt their activities by dynamically changing their parameters, or even their policies, to suit the changing system state (Willebeek-LeMair and Reeves, 1993).

The four policies that govern the action of a load balancing algorithm when a load imbalance is detected deal with information, transfer, location and selection. The information policy is responsible for keeping up-to-date load information about each resource in the system. The transfer policy deals with the dynamic aspects of a system. It uses the resources' load information to decide when a resource becomes eligible to act as a sender (transfer a job to another resource) or as a receiver (retrieve a job from another resource). Location policy selects partner resources for a job transfer transaction (Nandagopal and Rhymend Uthariaraj, 2011, Xu and Lau, 1994a). If the resource is an eligible sender, the location policy seeks out a receiver resource to receive the jobs. If the resource is an eligible receiver, the location policy looks for an eligible sender resource. Once a resource becomes an eligible sender or receiver, a selection policy is used to pick which of the queued jobs is to be transferred. A stable symmetrically initiated adaptive algorithm uses the information gathered during polling to classify the resources as sender/overloaded, receiver/underloaded or OK (resources having manageable load). The information about the state of the resources is maintained at each resource by a data structure composed of a sender list, a receiver list and an OK list. These lists are maintained using an efficient scheme and list-manipulative actions, such as moving a resource from one list to another or determining to which list a resource belongs. These actions impose a small and constant overhead, irrespective of the number of resources in the system. Consequently this algorithm scales well to large distributed systems (Willebeek-LeMair and Reeves, 1993). Selection policy uses several criteria to evaluate the queued jobs. Its goal is to select a job that reduces local load, incurs as little cost as possible in the transfer and has good affinity to the resource to which it is transferred.

Resource failures (processors/links) may frequently occur in grid systems and have an adverse effect on applications. Consequently, there is an increasing need for developing techniques to achieve fault tolerance. In multiprocessor systems, fault tolerance can be provided by scheduling replicas of jobs on different processors. There are two main approaches to replication, as described below:

  • (1)

    Active replication: This technique is based on space redundancy i.e. multiple copies of each job are mapped on different processors, which are run in parallel to tolerate a fixed number of failures. With such a technique, no fault detection mechanism is required.

  • (2)

    Passive replication: The main idea of this technique is that a backup job is activated only if a fault occurs while executing the primary job (Naedele, 1999, Oh and Son, 1997). It does not require fault diagnosis and is guaranteed to recover all affected jobs by processor failure. In such a scheme, only two copies of the job are scheduled on different processors (space exclusion and time exclusion). This approach is very useful for a grid where fault diagnosis is very difficult as one can discover a failure in a grid processor about what he/she could never know it is hardware platform model that has existed. Two techniques can be applied while scheduling primary and backup copies of each job. (1) Backup overloading consists of scheduling backups for multiple primary jobs during the same time slot in order to make efficient utilization of available processor time, and (2) De-allocation of resources reserved for backup jobs when the corresponding primaries complete successfully. All algorithms using these techniques (Naedele, 1999, Oh and Son, 1997, Alomari et al., 2004, Ghosh et al., 1997, Manimaran and Murthy, 1998, Qin and Jiang, 2006) share one or more of the common points: (i) jobs have deadlines and are independent, (ii) the system architecture is homogeneous and/or (iii) they support only one processor failure.

There has been a great effort in recent years in developing distributed load balancing algorithms, while trying to minimize all the communication needs resulting from the distributed nature. In this research, we have focused on designing distributed load balancing algorithm with the inspiration taken from neighbor-based and cluster-based load balancing schemes. Our main objective is to arrive at job assignments that will achieve minimum response time and minimum load difference between heaviest and lightest computing nodes thereby resulting in optimal computing node utilization. There are several objectives a good load balancer should address such as fairness, robustness and distribution; a detailed description of each is provided in Section 2. These requirements are addressed with the design of our algorithm.

Grosu and Chronopoulos (2005) and Penmatsa and Chronopoulos (2005) considered static load balancing in a system with servers and computers where servers balance load among all computers in a round robin fashion. It requires each server to have information on status of all computers as well as the load allocated by all other servers. Their work was extended to the multi-user case by Penmatsa and Chronopoulos. Penmatsa and Chronopoulos took into account communication delay and in Willebeek-LeMair and Reeves (1993) considered dynamic load balancing with the assumption that all jobs have the same execution time. Dhakal et al. (2007) studied dynamic load balancing in distributed computing systems and a stochastic model was developed, which takes into account randomness in delay. However, they do not consider the architecture where computers are located in different administrative domains. Shah et al. (2007) studied dynamic load balancing in grid systems using estimation on job arrival rate, processing rate and load on processor. However, they do not consider the architecture mentioned above and how to balance load in an optimal manner among computers. Arora et al. (2002) also studied dynamic load balancing in grid systems, however they did not consider resource unpredictability and how to balance load in an optimal manner among computers.

Xu et al. (1995) proposed a neighbor-based iterative dynamic load balancing algorithm for multi-computers in which processes migrate one step at a time—each step according to a local decision made by the intermediate processor concerned (Watts and Taylor, 1998). Lin and Keller (1987) presented a demand driven gradient model for load balancing (Hu and Blake, 1999). In this approach a job continues to transfer until it reaches an under-loaded computing node or it reaches a computing node for which no neighboring computing nodes reports shortest distance to the nearest lightly loaded computing node. Shu and KaleA (1989) put forward an adaptive contracting within neighborhood (ACWN) method, which applied a saturation control technique on the number of hops that the job had to travel before reaching its destination. ACWN performed consistently better than the gradient model but did not take into account a computing node's own load before making a job migration decision.

Cybenko (1989) first presented diffusion as a method for load balancing. Willebeek-LeMair and Reeves (1993) also explored diffusion and proved that it is superior to other load balancing strategies in terms of its performance, robustness and scalability. They assumed that no new workload is completed during execution of the algorithm. Xu and Lau, 1994a, Xu and Lau, 1994b derived optimal parameters that maximize the convergence rate on mesh, torus and N-D hypercube. Qian and Yang (1991) showed that without the quiescent assumption in Hui and Chanson (1999), it is possible only to prove that the variance of the unbalanced workload is bounded (Anand et al., 1999). J. Song (1994) proposed a partially asynchronous and iterative algorithm for distributed load balancing. A major disadvantage of diffusion approach is that it requires much iteration to achieve load balancing. Watts and Taylor (1998) overcame this using a fully implicit diffusion scheme with adaptive time steps. Another improved diffusion algorithm was derived based on Chebyshev polynomials and it showed significantly faster convergence than baseline diffusion method, but at the additional cost of calculating two Eigen values (Zhou et al., 1993).

Hui and Chanson, 1997, Hui and Chanson, 1999 proposed an intuitive approach based on a hydrodynamic analogy, for a heterogeneous environment characterized by different computing powers and uniform communication. Diekmann et al. (1999) proposed diffusion schemes for a computational environment characterized by uniform computing powers and different communication parameters (Nishimura et al., 2001). Elsasser et al. (2002) extended these schemes for computational environments that are heterogeneous both with respect to the processing performances and communication speeds. However Elsasser et al. (2002) neglected the overhead involved in collecting state information for load balancing. Anand, Ghose and Mani (1999) proposed a decentralized dynamic load balancing algorithm (ELISA). In this approach, the problem of frequent exchange of information is alleviated by estimating the load, based on system-state information received at sufficiently large intervals of time. The algorithm was designed to reduce communication delays by reducing the need for status exchange.

Arora et al. (2002) proposed a highly decentralized, sender-initiated and scalable algorithm for scheduling jobs and load balancing resources in heterogeneous grid environments. This approach is not applicable for system comprised of heterogeneous computing nodes separated by a wide-area broadband network.

In cluster based load balancing algorithms (Casanova and Marchal, 2002, Legrand et al., 2003, Wolski et al., 1999, Buyya and Murshed, 2002), the computing nodes are partitioned into clusters on the basis of network transfer delay. In Casanova and Marchal (2002) and Legrand et al. (2003) each cluster manager collects load information of all clusters and all computing nodes of its cluster for making load balancing decisions thereby introducing considerable communication overhead. In Wolski et al. (1999) the job migration cost is not considered for load balancing, also consulting the cluster manager for job dispatching results in non-negligible overhead and network delay. Naedele (1999) and Oh and Son (1997) propose a neighbor-based load balancing algorithm in which each cluster is limited to load information from within its own domain, which consists of itself and its neighbors. For low heterogeneous systems, the algorithm reduces load at weaker computing nodes and increases load at most powerful computing nodes. Alomari et al. (2004) and Ghosh et al. (1997) propose a decentralized load balancing algorithm for computational grid in which load updates between resources are done periodically, which leads to high messaging overhead.

In Song (1994) and Lu and Zomaya (2007) a sender-initiated decentralized dynamic load balancing algorithm is proposed for computational grid environment. Both authors juxtaposed the strong points of neighbor-based and cluster-based load balancing algorithms similar to us.

With respect to fault-tolerance, primary back-up approach was first studied in Budhiraja et al. (1992) where one server is selected as the primary and all the others are backups. If the primary fails, then a failover occurs and one of the backups takes over. In this paper, we consider a different approach where a backup is scheduled for each primary and they are located on two different processors. In this approach, a backup is executed when its primary cannot complete execution due to processor failure. Most works using the primary-backup approach (Ghosh et al., 1997, Manimaran and Murthy, 1998, AI-Omari et al., 2001, Abawajy, 2004, Oh and Son, 1997) consider scheduling of independent jobs. Backup overloading is introduced in Ghosh et al. (1997) to reduce replication cost of independent jobs, which allows scheduling backups of multiple primaries on the same or overlapping time interval on a processor.

Due to the emergence of grid computing over the Internet, there is presently a need for a hybrid load balancing algorithm, which takes into account grid architecture, computer heterogeneity, communication delay, network bandwidth, resource availability, resource unpredictability and job characteristics.

Taken altogether, the main contributions and novelties of our research are as follows:

  • (1)

    To the best of our knowledge, we are the first to tackle fault tolerant load balancing taking into account all the factors pertaining to the characteristics of the grid computing environment mentioned above.

  • (2)

    Our approach eliminates the complexity of a site to gather current state information of the whole grid since real time monitoring of whole grid will cause system overhead and is completely unrealistic in large scale grid environment.

  • (3)

    A well-designed information exchange scheduling scheme is adopted to enhance the efficiency of the load balancing model. We have proposed an extended version of the mutual information feedback policy specified in Song (1994) and Lu and Zomaya (2007).

The first-come first serve algorithm used in this article is similar to that proposed in Li et al. (2009). Remainder of this paper is organized as follows. The requirements for the design of distributed fault tolerant load balancing algorithm and its benefits are discussed in Section 2. It is followed by the overview of the system model. Section 3 presents our proposed algorithm in detail. We outline the principle of MCT and Min–min in Section 4. Section 5 focuses on the setup of the simulation and the experimental results. Performance criteria and environmental setting are introduced in this section and a thorough comparison of the performance of the algorithm with MCT and Min–min is provided. Finally, Section 6 is dedicated to conclusion and future work.

Section snippets

Design of the system environment

In this paper, a quite simple yet sufficiently realistic abstraction of grid computing environment is applied to carry out our work. In order to focus our attention on computing grids, we assume that the network connectivity involved is perfect. For experimental purposes we assume that the grid consists of a set of n sites =P1,P2,,Pn connected via different communication links with different speeds. In general, each site may contain multiple computing nodes (machines) and each computing

Design of hybrid load balancing policy

AlgHybrid_LB is carried out in two phases. In the first phase, a static load balancing policy selects the desired sites. If any site is unable to complete the assigned job, a new site will be located using the dynamic load balancing policy.

A brief description of MCT and Min–min

  • (1)

    Minimum completion time (MCT)

    MCT assigns each job, to the node, in an arbitrary order with the minimum expected completion time for that job (Braun et al., 2001). This method causes an assignment of some jobs to nodes that do not have minimum execution time for that job. The logic behind MCT is to combine the benefits of opportunistic load balancing (OLB) and minimum execution time (MET), while preventing the conditions in which OLB and MET perform poorly. This is much more successful

GridSim toolkit

The GridSim toolkit used as the simulation environment is a java-based discrete-event grid simulation toolkit. The toolkit supports modeling and simulation of heterogeneous grid resources (time-shared and space-shared), users and application models. It can also be used for the modeling and simulation of application scheduling on various classes of parallel and distributed computing systems such as clusters, grids and P2P networks.

The following are the reasons why the GridSim toolkit was chosen

Conclusion and future work

Fault tolerant load balancing is a crucial issue for the efficient operation of grid computing environments in distributing the jobs. Numerous items in the literature focus their attention on this and present a series of solutions. However, there are several drawbacks in the existing literature that are addressed by AlgHybrid_LB. The contributions of the designed algorithm can be categorized as follows: (1) Passive replication has not been combined with distributed load balancing in the grid

Acknowledgments

We thank the anonymous reviewers for their comments and recommendations, which have been crucial to improving the quality of this work.

References (55)

  • J. Song

    A partially asynchronous and iterative algorithm for distributed load balancing

    Parallel Computing

    (1994)
  • P. Sanders

    Analysis of nearest neighbor load balancing algorithms for random loads

    Parallel Computing

    (1999)
  • R. Wolski et al.

    The network weather service: a distributed resource performance forecasting service for metacomputing

    Journal of Future Generation Computing Systems

    (1999)
  • Kuo-Qin Yan et al.

    Towards a hybrid load balancing policy in grid computing system

    Journal of Expert Systems and Applications

    (2009)
  • Arora M, Das SK, Biswas R. A de-centralized scheduling and load balancing algorithm for heterogeneous Grid...
  • AI-Omari R, Somani AK, Maninaran G. A new fault-tolerant technique for improving schedulability in multiprocessor...
  • Abawajy JH. Fault-tolerant scheduling policy for grid computing systems. In: Proceedings of the international parallel...
  • R. Buyya et al.

    Gridsim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing

    Concurr Comput: Pract Exp

    (2002)
  • Budhiraja N, Marzullo K, Schneider FB, Toueg S. Primary-backup protocols: lower bounds and optimal implementations. In:...
  • H. Casanova et al.

    A network model for simulation of grid application

    Research Report 2002-40

    (2002)
  • S. Dhakal et al.

    Dynamic load balancing in distributed systems in the presence of delays: a regeneration theory approach

    IEEE Transactions on Parallel and Distributed Systems

    (2007)
  • R. Elsasser et al.

    Diffusion schemes for load balancing on heterogeneous networks

    Theory of Computing Systems

    (2002)
  • Feng Y, Li D, Wu H, Zhang Y. A dynamic load balancing algorithm based on distributed database system. In: Proceedings...
  • S. Ghosh et al.

    Fault-tolerance through scheduling of aperiodic tasks in hard real-time multiprocessor systems

    IEEE Trans Parallel and Distributed Systems

    (1997)
  • Grosan C, Abraham A, Helvik B. Multiobjective evolutionary algorithms for scheduling jobs on computational Grids. In:...
  • C.C. Hui et al.

    Hydrodynamic load balancing

    IEEE Transactions on Parallel and Distributed Systems

    (1999)
  • S. Hwang et al.

    A flexible framework for fault tolerance in the Grid

    Journal of Grid Computing

    (2003)
  • Cited by (54)

    • Optimal power allocation and load balancing for non-dedicated heterogeneous distributed embedded computing systems

      2019, Journal of Parallel and Distributed Computing
      Citation Excerpt :

      Our investigation makes a significant contribution to high-performance and energy-efficient computing in modern heterogeneous and distributed embedded systems. Load distribution and balancing in general distributed and parallel computing systems have been extensively studied, and a huge body of literature exists [3,15,25,29]. The majority of existing works on optimal load distribution have focused on performance metrics and system characteristics.

    • Load balancing in grid computing: Taxonomy, trends and opportunities

      2017, Journal of Network and Computer Applications
      Citation Excerpt :

      Both phases are designed to perform task distribution and an optimized rescheduling of distributed tasks. A hybrid policy for fault tolerant load balancing (HPFTLB) in grid computing environments (Balasangameshwara and Raju, 2012) is a cluster based load balancing strategy that is based on fault tolerance. In this strategy, the computing nodes or machines are part of each site, and single or multiple processing elements with different processing power are part of each machine.

    • Survey of load balancing techniques for Grid

      2016, Journal of Network and Computer Applications
    • A load-balanced hybrid heuristic for allocation of batch of tasks in cloud computing environment

      2023, International Journal of Pervasive Computing and Communications
    View all citing articles on Scopus
    View full text