Continuous OptimizationImproving an interior-point approach for large block-angular problems by hybrid preconditioners
Introduction
Many important large-scale optimization problems exhibit a block-angular structure. Applications are found in fields such as control and planning, network flows, stochastic linear programming, and statistical data protection. Several interior-point methods have been devised to solve these structured problems [5], [7], [12], [16], [25]. These specialized algorithms exploit the particular structure of the constraints matrix, and some were implemented for parallel environments [5], [25]. The efficiency of interior-point methods critically depends of the linear system solver used at each iteration to compute the Newton direction. Such systems are often written in a symmetric indefinite form, known as the augmented system. They can also be reduced to a smaller positive definite form, the normal equations. Techniques based on direct and iterative solvers can be applied for their solution. For some classes of large scale problems the use of direct methods becomes prohibitive due to storage and time limitations, whereas iterative linear solvers with appropriate preconditioners may be more efficient.
The efficient interior-point algorithm for primal block-angular problems of [15] solved the normal equations in two stages: Cholesky factorizations for the block constraints and a Preconditioned Conjugate Gradient (PCG) for the linking constraints. The purpose of PCG is to avoid solving the system associated to the complicating linking constraints by Cholesky factorizations, in an attempt to make the problem block separable. The preconditioner is obtained by truncating an infinite power series that approximates the inverse of the system to be solved. For some difficult primal block-angular problems this approach outperformed state-of-the-art commercial solvers [16]. However, in some problems, systems become very ill-conditioned as the optimal solution is reached, and then PCG provides slow and inaccurate solutions. It was shown [16] that the efficiency of this approach depends on the spectral radius—in [0, 1)—of a certain matrix which appears in the definition of the preconditioner (which is itself related to the Schur complement of the normal equations). Spectral radius close to 1 degrades the performance of the preconditioner. When PCG gives inaccurate solutions, the code implemented in [15] switches to the solution of the normal equations by a Cholesky factorization, which may be prohibitive for large-scale problems.
In order to yield a reliable and efficient interior-point method based just on iterative solvers we introduce a hybrid and adaptive scheme for solving the normal equations. On the first interior-point iterations the normal equations are solved using the Cholesky-PCG approach of [15] outlined above. When the system associated to linking constraints becomes ill-conditioned, the normal equations are solved by a PCG using the splitting preconditioner of [29], [30], instead of switching to a direct solver. The splitting preconditioner is a generalization of the tree preconditioner of [33] for large-scale minimum cost network flow problems. Based on a LU factorization, the splitting preconditioner was specially tailored for the last interior-point iterations, when the systems are ill-conditioned. We developed a new and efficient criterion to identify when (i.e., at which interior-point iteration) to switch between iterative solvers. This criterion is based on both the Ritz values of the matrix that appears in the definition of the power series preconditioner, and the number of PCG iterations needed at each interior-point iteration. The Ritz values are approximations of the eigenvalues of a matrix; they will be used to estimate the spectral radius, which measures the efficiency of the power series preconditioner. An implementation of this new approach, combining the power series and the splitting preconditioners, was applied to three classes of primal block-angular instances [15]: multicommodity flows (oriented and nonoriented), minimum-distance controlled tabular adjustment for statistical data protection, and the minimum congestion problem. As it will be shown, the hybrid approach was more efficient than the power series preconditioner in many block-angular problems. Other hybrid approaches combining interior-point and combinatorial algorithms have been used for some type of networks flows problems [21].
This paper is organized as follows. In Section 2 we recall the basic ideas of interior-point methods for primal block-angular problems using the power series preconditioner. The new hybrid approach is described in Section 3, together with an outline of the splitting preconditioner, and a description of the switching criterion between preconditioners. Numerical experiments are shown in Section 4. The effect of different regularization parameters for the splitting preconditioner are also discussed in Section 4. Finally, in Section 5 the conclusions are drawn and further developments are suggested.
Section snippets
The interior-point algorithm for primal block-angular problems
One of the most efficient interior-point methods for some classes of block-angular problems was initially developed for multicommodity flows [12] and later extended for general primal block-angular problems [15]. This method considers the following general formulation of a block-angular problem:Matrices and , define, respectively, the block and linking constraints, k being the number of blocks.
The hybrid approach for normal equations
The hybrid approach works as follows. Initially, the normal equations are solved by the procedure described in Section 2.1, i.e., solving (11) by Cholesky factorizations and (10) by PCG with the power series preconditioner. When the power series preconditioner becomes inefficient, then the method switches to the solution of the normal Eq. (7) by PCG with the splitting preconditioner [30]. The splitting preconditioner and the efficient criteria developed to identify the switch between
Numerical experiments
The hybrid approach described in the previous section has been added to a MATLAB implementation of the specialized algorithm for general block-angular problems, named BlockIP [15]. BlockIP implements a standard infeasible primal–dual path-following algorithm, which solves the normal equations by either the specialized procedure described in Section 2.1 or a Cholesky factorization. The code uses the Ng-Peyton sparse Cholesky package [22], [28] for the solution of (11), (7); the Ng-Peyton sparse
Conclusions
We have provided computational evidence that the hybrid approach combined with a new switch criterion significantly improved the performance of the specialized interior-point algorithm for some classes of primal block-angular problems. An estimate for the spectral radius of the matrix D−1(CTB−1C) was computed by using the Ritz values. This resulted in a criterion to switch between preconditioners that worked fine in the tested instances.
Improving the efficiency of the PCG by an adaptive
Acknowledgments
This work was developed when the first author was visiting the Department of Statistics and Operations Research of the Universitat Politècnica de Catalunya, funded by the CAPES/Fundação Carolina, Brazil. The second author has been supported by Grants MTM2009-08747 and MTM2012-31440 of the Spanish research program, and SGR-2009-1122 of the Government of Catalonia. The third author research is supported by CNPq and FAPESP. The authors thank F.F. Campos for suggesting the use of Ritz values.
References (38)
- et al.
Parallel algorithms to solve two-stage stochastic linear programs with robustness constraints
Parallel Computing for Irregular Applications
(2000) Minimum-distance controlled perturbation methods for large-scale tabular data protection
European Journal of Operational Research
(2006)- et al.
Combining a hybrid preconditioner and a optimal adjustment algorithm to accelerate the convergence of interior point methods
Linear Algebra and its Applications
(2012) - et al.
A new class of preconditioners for large-scale linear systems from interior point methods for linear programming
Linear Algebra and its Applications
(2005) - et al.
Further results on the convergence behavior of conjugate-gradients and Ritz values
Linear Algebra and its Applications
(1996) - A. Ali and J.L. Kennington. Mnetgen program documentation. Technical Report 77003, Department of Industrial Engineering...
- et al.
Preconditioning indefinite systems in interior point methods for large scale linear optimisation
Optimization Methods & Software
(2008) - et al.
Regularized symmetric indefinite systems in interior point methods for linear and quadratic optimization
Optimization Methods & Software
(1999) - et al.
LAPACK Users’ Guide
(1999) - et al.
Asymptotic analysis of the flow deviation method for the maximum concurrent flow problem
Mathematical Programming
(2002)