EURASIP Journal on Applied Signal Processing 
Volume 2006 (2006), Article ID 26318, 19 pages
doi:10.1155/ASP/2006/26318

Denoising by Sparse Approximation: Error Bounds Based on Rate-Distortion Theory

Alyson K. Fletcher,1 Sundeep Rangan,2 Vivek K Goyal,3 and Kannan Ramchandran4

 1Department of Electrical Engineering and Computer Sciences, University of California, Berkeley 94720-1770, CA, USA
2Flarion Technologies Inc., Bedminster 07921, NJ, USA
3Department of Electrical Engineering and Computer Science and Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge 02139-4307, MA, USA
4Department of Electrical Engineering and Computer Sciences, College of Engineering, University of California, Berkeley 94720-1770, CA, USA
Received 9 September 2004; Revised 6 June 2005; Accepted 30 June 2005

Abstract

If a signal x is known to have a sparse representation with respect to a frame, it can be estimated from a noise-corrupted observation y by finding the best sparse approximation to y. Removing noise in this manner depends on the frame efficiently representing the signal while it inefficiently represents the noise. The mean-squared error (MSE) of this denoising scheme and the probability that the estimate has the same sparsity pattern as the original signal are analyzed. First an MSE bound that depends on a new bound on approximating a Gaussian signal as a linear combination of elements of an overcomplete dictionary is given. Further analyses are for dictionaries generated randomly according to a spherically-symmetric distribution and signals expressible with single dictionary elements. Easily-computed approximations for the probability of selecting the correct dictionary element and the MSE are given. Asymptotic expressions reveal a critical input signal-to-noise ratio for signal recovery.