Dynamic load balancing for parallel interval-Newton using message passing
Introduction
Process systems engineering involves the development and application of computer-aided, model-based techniques for the design, optimization, operation and control of chemical processing systems, as well as for the associated planning, scheduling and maintenance problems. In this field, two core numerical issues are the solution of nonlinear equation systems and of nonlinear programming (optimization) problems. However, neither of these problems can be solved with complete reliability using standard techniques. In this context, it has been shown (e.g. Balaji & Seader, 1995, Berner, McKinnon & Millar, 1999, Byrne & Bogle, 1996, Byrne & Bogle, 2000, Gau, Brennecke & Stadtherr, 2000, Gau & Stadtherr, 2000, Han, Manousiousthakis & Choi, 1997; Hua et al., 1996, Hua et al., 1998; Maier et al., 1998, Maier et al., 2000; Maier & Stadtherr, 2001, McKinnon, Millar & Mongeau, 1996, Schnepper & Stadtherr, 1996, Stadtherr, Schnepper & Brennecke, 1995, Stradi, Brennecke, Kohn & Stadtherr, 2001, Tessier, Brennecke & Stadtherr, 2000, Vaidyanathan & El-Halwagi, 1994, Xu, Scurto, Castier, Brennecke & Stadtherr, 2000) that methods based on interval mathematics, such as the interval-Newton technique (e.g. Hansen, 1992, Kearfott, 1996, Neumaier, 1990), can be used to solve the nonlinear equation solving and optimization problems with complete reliability, providing a mathematical and computational guarantee that either all roots are found in the nonlinear equation solving problem, or that the global optimum is found in the nonlinear programming problem. Unfortunately, though not unexpectedly, this guarantee of reliability may come at the cost of a high CPU time requirement, though it is often difficult to predict from problem characteristics, such as number of variables, when this is going to be the case. Fortunately, the interval-Newton method is implemented using branch-and-prune (BP) or branch-and-bound (BB) strategies that present a natural parallelism, and which therefore are particularly amenable to parallel processing. Thus, we seek to make effective use of high performance computing technology (parallel computing) to implement BP and BB strategies for reliably and efficiently solving problems in chemical process modeling and engineering.
Of particular interest here is the use of distributed parallel computing using a cluster of workstations (COW), in which multiple workstations on a network are used as a single parallel computing resource by employing message passing. This sort of parallel computing system has advantages since it is relatively inexpensive, and is based on widely available hardware. Thus, such an approach to parallel computing has become an important trend in providing high performance computing resources in science and engineering. In this paper, we focus specifically on issues of load balancing and scheduling that arise in the implementation of parallel BB and BP using message passing on a workstation cluster, and describe and analyze techniques for this purpose. In particular, we will consider load balancing and scheduling in the context of an interval-Newton/generalized-bisection (IN/GB) algorithm, details of which are given by Schnepper and Stadtherr (1996). Applications to problems arising in chemical process engineering are used to demonstrate the effectiveness of the approach used. These example problems are: (1) estimation of parameters in the Wilson equation using a maximum likelihood objective for a mixture of formic acid and water (Gau, 2001); (2) computation of mixture critical points from cubic equation-of-state models (Stradi et al., 2001); and (3) estimation of parameters in the Van Laar equation using an error-in-variables approach for a mixture of methanol and 1,2-dichloroethane (Esposito & Floudas, 1998, Gau & Stadtherr, 2000).
Section snippets
Background
BP and BB algorithms are general-purpose intelligent search techniques for finding all solutions, or the optimal solution, within a search space of interest, and have a wide range of applications. These techniques employ successive decomposition (tessellation) of the global problem into smaller disjoint or independent subproblems that are solved recursively until all solutions, or the optimal solution, are found. BB and BP methods have many important applications in engineering and science,
Dynamic load balancing
As noted above, since the subproblems to be solved are independent, the execution of interval-Newton techniques, whether BP or BB, on distributed parallel systems can clearly provide improvements in computational efficiency. And since, for some practical problems, the binary tree that needs to be searched may be quite large, there may in fact be a strong motivation for trying to exploit the opportunity for parallel computing. However, because of the irregular structure of the binary tree, doing
Implementation of dynamic load balancing algorithms
In this section, a sequence of three algorithms (SWS, SDLB and ADLB) is described for load balancing in a binary tree, with each algorithm in the sequence representing an improvement in principle over the previous one. The last method (ADLB) represents a synthesis of the most attractive and effective strategies adapted from previous research studies, and also incorporates some novel strategies in this context. Though ADLB is expected to be superior, we describe all three approaches, as this
Effect of virtual network
The virtual network used in the local coordination of neighbor processors for workload distribution and message propagation has an important effect on network communication overhead. Since more connections mean higher communication costs, the number of neighbors on the virtual network affects the message propagation speed across the set of processors. On one hand, having more connections among processors can reduce the message diffusion distance and help balance overloaded or underloaded
Stack management
Another issue of interest is how to improve the parallel search efficiency of interval BB for global optimization problems. As noted above, when a parallel BB approach is applied to solve global minimization problems, the amount of work required will typically differ depending on the number of processors used. This is because the number of processors used affects the timing with which best (least) upper bounds on the global minimum are found and propagated across the network to the entire
Concluding remarks
We have described here how load management strategies can be used for effectively solving interval BB and BP problems in parallel on a network-based COW. Of the dynamic load balancing algorithms considered, the best performance was achieved by the ADLB approach. This overlaps communication and computation by the use of the asynchronous nonblocking communication functions provided by MPI, and uses a type of diffusive load-adjusting scheme to prevents out-of-work idle states while keeping
Acknowledgements
This work has been supported in part by the donors of The Petroleum Research Fund, administered by the ACS, under Grant 35979-AC9, by the National Science Foundation Grant EEC97-00537-CRCD, by the Environmental Protection Agency Grant R826-734-01-0. Computational hardware was provided in part through US Army Research Office Grant DAAG55-98-1-0091.
References (83)
- et al.
Stochastic optimization based algorithms for process synthesis under uncertainty
Computers & Chemical Engineering
(1998) - et al.
An accelerated interval method for global optimization
Computers & Chemical Engineering
(1996) A parallel approximation scheme for the multiprocessor scheduling problem
Parallel Computing
(2000)- et al.
Derivation of a termination detection algorithm for distributed computations
Information Processing Letters
(1983) - et al.
Termination detection for diffusing computations
Information Processing Letters
(1980) - et al.
The use of dynamic programming with parallel computers for process synthesis
Computers & Chemical Engineering
(1994) - et al.
Portable code for process synthesis using workstation clusters and distributed memory multicomputers
Computers & Chemical Engineering
(1995) - et al.
A general matrix iterative model for dynamic load balancing
Parallel Computing
(1996) - et al.
Parallel all-row preconditioned interval linear solver for nonlinear equations on multiprocessors
Parallel Computing
(1994) - et al.
Reliable nonlinear parameter estimation using interval analysis: error-in-variable approach
Computers & Chemical Engineering
(2000)
Reliable parameter estimation in VLE modeling
Fluid Phase Equilibria
Reliable prediction of phase stability using an interval-Newton method
Fluid Phase Equilibria
Deterministic global optimization and torsion angle dynamics for molecular structure prediction
Computers and Chemical Engineering
A distributed control architecture for branch and bound calculations
Computers & Chemical Engineering
Reliable computation of reactive azeotropes
Computers & Chemical Engineering
GLOPEQ: a new computational tool for the phase and chemical equilibrium problem
Computers & Chemical Engineering
A distributed processing algorithm for solving integer problems using a cluster of workstations
Parallel Computing
An exact parallel algorithm for scheduling when production costs depend on consecutive system states
Computers & Chemical Engineering
An exact algorithm for resource constrained sequencing with application to production scheduling under an aggregate deadline
Computers & Chemical Engineering
Experiments with parallel branch-and-bound algorithms for the set covering problems
Operations Research Letters
Robust process simulation using interval methods
Computers & Chemical Engineering
A symbolic reformulation/spatial branch-and-bound algorithm for the global optimization of nonconvex MINLPs
Computers & Chemical Engineering
Adaptive parallel algorithms for integral knapsack problems
Journal of Parallel and Distributed Computing
Reliable phase stability analysis for excess Gibbs energy models
Chemical Engineering Science
A study of parallel branch-and-bound algorithms with best-bound-first search
Parallel Computing
Global optimization of mixed-integer nonlinear problems
AIChE Journal
Distributed branch and bound algorithms for global optimization
Distributed decomposition based approaches in global optimization
Application of interval-Newton method to chemical engineering problems
AIChE Symposium Series
Nonlinear parameter estimation
A parallel algorithm for the global optimization of Gibbs free energy
Annals of Operations Research
Scheduling multithreaded computations by work stealing
Journal of the ACM
Global optimization of modular process flowsheets
Industrial and Engineering Chemistry Research
Implementations of parallel branch-and-bound algorithms-experience with the graph partitioning problem
Annals of Operations Research
Parallel best-first branch-and-bound in discrete optimization: a framework
Dynamic load balancing for distributed memory multiprocessors
Journal of Parallel and Distributed Computing
Adapative load sharing in homogeneous distributed systems
IEEE Transactions on Software Engineering
Global optimization in parameter estimation of nonlinear algebraic models via the error-in-variables approach
Industrial and Engineering Chemistrty Research
Cited by (14)
Global optimization in the 21st century: Advances and challenges
2005, Computers and Chemical EngineeringReliable computation of equilibrium states and bifurcations in food chain models
2004, Computers and Chemical EngineeringGlobal optimization in the 21st century: Advances and challenges
2004, Computer Aided Chemical EngineeringInterval Methods for Solving Various Kinds of Quantified Nonlinear Problems
2020, Studies in Computational IntelligenceParallelization of B&BT Algorithms
2019, Studies in Computational IntelligenceBranch-and-bound-type methods
2019, Studies in Computational Intelligence
- 1
Present address: LINDO Systems, Inc., 1415 North Dayton Street, Chicago, IL 60622, USA.