Continuous Optimization
Improving an interior-point approach for large block-angular problems by hybrid preconditioners

https://doi.org/10.1016/j.ejor.2013.04.007Get rights and content

Highlights

  • Interior-point method for block-angular problems based on hybrid approach.

  • Hybrid preconditioner combining power series and splitting preconditioners.

  • New switching criteria between preconditioners based on Ritz values.

  • Computational results provided for three classes of problems.

  • The hybrid approach resulted more efficient for most instances.

Abstract

The computational time required by interior-point methods is often dominated by the solution of linear systems of equations. An efficient specialized interior-point algorithm for primal block-angular problems has been used to solve these systems by combining Cholesky factorizations for the block constraints and a conjugate gradient based on a power series preconditioner for the linking constraints. In some problems this power series preconditioner resulted to be inefficient on the last interior-point iterations, when the systems became ill-conditioned. In this work this approach is combined with a splitting preconditioner based on LU factorization, which works well for the last interior-point iterations. Computational results are provided for three classes of problems: multicommodity flows (oriented and nonoriented), minimum-distance controlled tabular adjustment for statistical data protection, and the minimum congestion problem. The results show that, in most cases, the hybrid preconditioner improves the performance and robustness of the interior-point solver. In particular, for some block-angular problems the solution time is reduced by a factor of 10.

Introduction

Many important large-scale optimization problems exhibit a block-angular structure. Applications are found in fields such as control and planning, network flows, stochastic linear programming, and statistical data protection. Several interior-point methods have been devised to solve these structured problems [5], [7], [12], [16], [25]. These specialized algorithms exploit the particular structure of the constraints matrix, and some were implemented for parallel environments [5], [25]. The efficiency of interior-point methods critically depends of the linear system solver used at each iteration to compute the Newton direction. Such systems are often written in a symmetric indefinite form, known as the augmented system. They can also be reduced to a smaller positive definite form, the normal equations. Techniques based on direct and iterative solvers can be applied for their solution. For some classes of large scale problems the use of direct methods becomes prohibitive due to storage and time limitations, whereas iterative linear solvers with appropriate preconditioners may be more efficient.

The efficient interior-point algorithm for primal block-angular problems of [15] solved the normal equations in two stages: Cholesky factorizations for the block constraints and a Preconditioned Conjugate Gradient (PCG) for the linking constraints. The purpose of PCG is to avoid solving the system associated to the complicating linking constraints by Cholesky factorizations, in an attempt to make the problem block separable. The preconditioner is obtained by truncating an infinite power series that approximates the inverse of the system to be solved. For some difficult primal block-angular problems this approach outperformed state-of-the-art commercial solvers [16]. However, in some problems, systems become very ill-conditioned as the optimal solution is reached, and then PCG provides slow and inaccurate solutions. It was shown [16] that the efficiency of this approach depends on the spectral radius—in [0, 1)—of a certain matrix which appears in the definition of the preconditioner (which is itself related to the Schur complement of the normal equations). Spectral radius close to 1 degrades the performance of the preconditioner. When PCG gives inaccurate solutions, the code implemented in [15] switches to the solution of the normal equations by a Cholesky factorization, which may be prohibitive for large-scale problems.

In order to yield a reliable and efficient interior-point method based just on iterative solvers we introduce a hybrid and adaptive scheme for solving the normal equations. On the first interior-point iterations the normal equations are solved using the Cholesky-PCG approach of [15] outlined above. When the system associated to linking constraints becomes ill-conditioned, the normal equations are solved by a PCG using the splitting preconditioner of [29], [30], instead of switching to a direct solver. The splitting preconditioner is a generalization of the tree preconditioner of [33] for large-scale minimum cost network flow problems. Based on a LU factorization, the splitting preconditioner was specially tailored for the last interior-point iterations, when the systems are ill-conditioned. We developed a new and efficient criterion to identify when (i.e., at which interior-point iteration) to switch between iterative solvers. This criterion is based on both the Ritz values of the matrix that appears in the definition of the power series preconditioner, and the number of PCG iterations needed at each interior-point iteration. The Ritz values are approximations of the eigenvalues of a matrix; they will be used to estimate the spectral radius, which measures the efficiency of the power series preconditioner. An implementation of this new approach, combining the power series and the splitting preconditioners, was applied to three classes of primal block-angular instances [15]: multicommodity flows (oriented and nonoriented), minimum-distance controlled tabular adjustment for statistical data protection, and the minimum congestion problem. As it will be shown, the hybrid approach was more efficient than the power series preconditioner in many block-angular problems. Other hybrid approaches combining interior-point and combinatorial algorithms have been used for some type of networks flows problems [21].

This paper is organized as follows. In Section 2 we recall the basic ideas of interior-point methods for primal block-angular problems using the power series preconditioner. The new hybrid approach is described in Section 3, together with an outline of the splitting preconditioner, and a description of the switching criterion between preconditioners. Numerical experiments are shown in Section 4. The effect of different regularization parameters for the splitting preconditioner are also discussed in Section 4. Finally, in Section 5 the conclusions are drawn and further developments are suggested.

Section snippets

The interior-point algorithm for primal block-angular problems

One of the most efficient interior-point methods for some classes of block-angular problems was initially developed for multicommodity flows [12] and later extended for general primal block-angular problems [15]. This method considers the following general formulation of a block-angular problem:mini=ok(ciTxi+xiTQixi)s.t.N1N2NkL1L2LkIx1x2xkx0=b1b2bkb00xiuii=1,,k.Matrices NiRmi×ni and LiRl×ni,i=1,,k, define, respectively, the block and linking constraints, k being the number of blocks.

The hybrid approach for normal equations

The hybrid approach works as follows. Initially, the normal equations are solved by the procedure described in Section 2.1, i.e., solving (11) by Cholesky factorizations and (10) by PCG with the power series preconditioner. When the power series preconditioner becomes inefficient, then the method switches to the solution of the normal Eq. (7) by PCG with the splitting preconditioner [30]. The splitting preconditioner and the efficient criteria developed to identify the switch between

Numerical experiments

The hybrid approach described in the previous section has been added to a MATLAB implementation of the specialized algorithm for general block-angular problems, named BlockIP [15]. BlockIP implements a standard infeasible primal–dual path-following algorithm, which solves the normal equations by either the specialized procedure described in Section 2.1 or a Cholesky factorization. The code uses the Ng-Peyton sparse Cholesky package [22], [28] for the solution of (11), (7); the Ng-Peyton sparse

Conclusions

We have provided computational evidence that the hybrid approach combined with a new switch criterion significantly improved the performance of the specialized interior-point algorithm for some classes of primal block-angular problems. An estimate for the spectral radius of the matrix D−1(CTB−1C) was computed by using the Ritz values. This resulted in a criterion to switch between preconditioners that worked fine in the tested instances.

Improving the efficiency of the PCG by an adaptive

Acknowledgments

This work was developed when the first author was visiting the Department of Statistics and Operations Research of the Universitat Politècnica de Catalunya, funded by the CAPES/Fundação Carolina, Brazil. The second author has been supported by Grants MTM2009-08747 and MTM2012-31440 of the Spanish research program, and SGR-2009-1122 of the Government of Catalonia. The third author research is supported by CNPq and FAPESP. The authors thank F.F. Campos for suggesting the use of Ritz values.

References (38)

  • J.R. Birge et al.

    Computing block angular Karmarkar projections with applications to stochastic programming

    Management Science

    (1988)
  • S. Bocanegra et al.

    Using a hybrid preconditioner for solving large-scale linear systems arising from interior point methods

    Computational Optimization and Applications

    (2007)
  • E.G. Boman et al.

    Maximum-weight-basis preconditioners

    Numerical Linear Algebra with Applications

    (2004)
  • J. Brandts et al.

    The convergence of Krylov methods and Ritz values

    Conjugate Gradient Algorithms and Finite Element Methods. Scientific Computation

    (2004)
  • W.J. Carolan et al.

    An empirical evaluation of the KORBX algorithms for military airlift applications

    Operations Research

    (1990)
  • J. Castro

    A specialized interior-point algorithm for multicommodity network flows

    SIAM Journal on Optimization

    (2000)
  • J. Castro

    Quadratic interior-point methods in statistical disclosure control

    Computational Management Science

    (2005)
  • J. Castro

    An interior-point approach for primal block-angular problems

    Computational Optimization and Applications

    (2007)
  • J. Castro et al.

    Quadratic regularizations in an interior-point method for primal block-angular problems

    Mathematical Programming

    (2011)
  • Cited by (0)

    View full text