Abstract

A simultaneous sparse approximation problem requests a good approximation of several input signals at once using different linear combinations of the same elementary signals. At the same time, the problem balances the error in approximation against the total number of elementary signals that participate. These elementary signals typically model coherent structures in the input signals, and they are chosen from a large, linearly dependent collection.

The first part of this paper proposes a greedy pursuit algorithm, called simultaneous orthogonal matching pursuit (S-OMP), for simultaneous sparse approximation. Then it presents some numerical experiments that demonstrate how a sparse model for the input signals can be identified more reliably given several input signals. Afterward, the paper proves that the S-OMP algorithm can compute provably good solutions to several simultaneous sparse approximation problems.

The second part of the paper develops another algorithmic approach called convex relaxation, and it provides theoretical results on the performance of convex relaxation for simultaneous sparse approximation.

Keywords: Greedy algorithms; Orthogonal matching pursuit; Multiple measurement vectors; Simultaneous sparse approximation; Subset selection

Article Outline

1. Introduction

1.1. Contributions

1.2. Outline

2. Background

2.1. Signal matrices

2.2. The dictionary

2.3. Coherence

2.4. Coefficient matrices

2.5. Cost of approximation

2.6. Vector and matrix norms

3. Simultaneous orthogonal matching pursuit

3.1. Statement of algorithm

3.2. Stopping criteria

4. Numerical experiments

5. Performance guarantees for S-OMP

5.1. Approximation with a sparsity bound

5.2. Approximation with an error bound

5.3. Approximation with a correlation bound

5.4. Proof of Theorem 5.3

6. Comparison with previous work

References