Elsevier

Linear Algebra and its Applications

Volume 484, 1 November 2015, Pages 322-343
Linear Algebra and its Applications

Randomized block Kaczmarz method with projection for solving least squares

https://doi.org/10.1016/j.laa.2015.06.027Get rights and content
Under an Elsevier user license
open archive

Abstract

The Kaczmarz method is an iterative method for solving overcomplete linear systems of equations Ax=b. The randomized version of the Kaczmarz method put forth by Strohmer and Vershynin iteratively projects onto a randomly chosen solution space given by a single row of the matrix A and converges linearly1 in expectation to the solution of a consistent system. In this paper we analyze two block versions of the method each with a randomized projection, designed to converge in expectation to the least squares solution, often faster than the standard variants. Our approach utilizes both a row and column-paving of the matrix A to guarantee linear convergence when the matrix has consistent row norms (called nearly standardized), and a single column-paving when the row norms are unbounded. The proposed methods are an extension of the block Kaczmarz method analyzed by Needell and Tropp and the Randomized Extended Kaczmarz method of Zouzias and Freris. The contribution is thus two-fold; unlike the standard Kaczmarz method, our results demonstrate convergence to the least squares solution of inconsistent systems (both methods in the nearly standardized case and the second method in other cases). By using appropriate blocks of the matrix this convergence can be significantly accelerated, as is demonstrated by numerical experiments.

MSC

65F10
65F20
68W20
41A65

Keywords

Block Kaczmarz
Randomized Extended Kaczmarz
Projections onto convex sets
Algebraic reconstruction technique
Matrix paving

Cited by (0)