Journal of Visual Communication and Image Representation
Image postprocessing by Non-local Kuan’s filter
Research highlights
► A DCT domain Non-local Kuan’s filter is proposed. ► The required assumptions for the filter are analyzed theoretically. ► Dual layer filtering process is used to restore DCT coefficients. ► Blocking artifacts of coded images are efficiently removed.
Introduction
Block-based discrete cosine transform (DCT) coding is adopted in several industry standards, such as JPEG, MPEG, and H.264, for image and video compression. In a block-based DCT scheme, an image is firstly divided into non-overlapping blocks and pixels in each block are transformed into the DCT coefficients which are then quantized. At low bit rates, blocking artifacts will be visible, because quantization is performed independently at each block without considering the existing correlations among adjacent blocks. The use of a postprocessing technique on a coded image is a common strategy to reduce blocking artifacts and improve fidelity, as it does not require modifications of existing image/video standards and so is readily available. Based on different types of postprocessing techniques, such as post-filtering, iterated projection onto convex sets (POCS), maximum a posteriori (MAP) estimation, overcomplete wavelet, a variety of postprocessing methods have been developed to cope with the problem.
Early attempts used image enhancement approach. For example, Lim and Reeve [1] applied low-pass filtering to pixels along block boundaries. This method sometimes blurs true edges, so some adaptive post-filtering techniques were proposed. Ramamurthi and Gersho [2] applied 2D low-pass filtering only to areas away from edges and performed 1D filtering to areas near edges to avoid blurring them. The post-filtering techniques used in recent coding standards such as H.264/AVC [3] and MPEG-4 [4] consist of several filters and one of them is selected based on local activity. The abovementioned methods are performed in the spatial domain. Recently, some post-filtering techniques in the DCT domain were proposed [5], [6], [7], which have demonstrated that postprocessing in transform domain is a promising postprocessing approach.
Methods based on POCS techniques [8], [9], [10], [11], [12] were also proposed. The idea is to impose smoothness constraint around block boundaries and project block images onto these convex sets (POCS) iteratively. Both forward and inverse DCT are required in each iteration, so the major drawback is high computational complexity. In order to reduce computation, Gan et al. [12] proposed a novel smoothness constraint set in the DCT domain and used the iterative POCS technique to reduce blocking artifacts.
By formulating postprocessing as an inverse problem, the MAP estimation technique can be used to find the solution. The probability function of the original image is modeled by different Markov random field (MRF) models [13], [14], [15], [16], [17]. Sun and Cham [16], [17] modeled the original image as a high order MRF using the Fields of Experts (FoE) framework, and proposed an effective postprocessing method by MAP criterion. Li and Delp [18] addressed the problem using a DCT domain MRF model to do the MAP estimation.
As to the overcomplete wavelet technique [19], [20], [21], it uses the overcomplete wavelet representation to reduce blocking artifacts. Xiong et al. [19] used thresholding of the overcomplete wavelet coefficients to reduce the quantization effects. In [20] the wavelet transform modulus maxima representation was adopted for deblocking. Liew and Yan [21] analyzed the block discontinuities caused by coding to derive more accurate thresholds at different wavelet scales. Recently, a novel deblocking method based on the shape-adaptive DCT [22] was developed, which achieved the best performance until now.
Kuan’s filter [24] can produce the linear minimum mean-square-error (MMSE) for a signal corrupted with uncorrelated, signal-dependent noise. We develop a more accurate DCT domain Kuan’s filter using Non-local parameter estimation technique. Two assumptions are needed to derive the solution of the filter. We investigate the validity of the assumptions and verify each of them. The DCT domain Kuan’s filter is applied for low frequency DCT coefficients that satisfy the two assumptions and Non-local mean filter is used for high frequency AC coefficients. Then we have the proposed Non-local Kuan’s (NLK) filter. A new image postprocessing method called Dual Non-local Kuan’s (DNLK) filter is then proposed. It uses the NLK filter to obtain good estimates of image statistics and then applies the NLK filter again to reduce quantization noise. We also found that the proposed NLK filter can be used for the estimation of the overcomplete DCT coefficients. Hence, the DNLK filter is combined with the overcomplete representation to form the Overcomplete Dual Non-local Kuan’s (OCDNLK) filter. Experimental results show that the OCDNLK filter, in most cases, can achieve higher PSNR gain than other state-of-the-arts methods and generates images with the best visual quality. The performance of the DNLK filter, which has lower complexity than the OCDNLK filter, is comparable to other state-of-the-arts methods. Moreover, we demonstrate the efficiency of the two methods on JPEG coded images under various image quality settings.
In Section 2, we explain the proposed Kuan’s filter in the DCT domain using Non-local parameter estimation technique which achieves LMMSE, analyze the two assumptions required, and finally develop the NLK filter used for all DCT coefficients. Section 3 presents the two image postprocessing methods, DNLK filter and OCDNLK filter. Experimental results on images using test quantization tables and JPEG coded images are given in Section 4. Finally we draw conclusions in Section 5.
Section snippets
The proposed Non-local Kuan’s filter in the DCT domain
A roundoff quantizer is a non-linear device having a staircase-type input–output characteristic as shown in Fig. 1. At the transmitting end, if we know the original DCT coefficients x0 and quantization step Q, the quantized value y0 ϵ {rQ ; r = 0, ±1, ±2, …} and quantization error n will be determined. However, at the receiving end, we have the quantized value y0, and only know that error n is in the range [−Q/2, Q/2] and the input x0 is in the range [y0 − Q/2, y0 + Q/2]. Let y0 = x0 + n, where n is the
Image postprocessing by the dual Non-local Kuan’s filter
In Section 2, we have described the proposed NLK filter and methods for estimation of its parameters. In this section, we first propose a postprocessing method called Dual Non-local Kuan’s (DNLK) filter by applying this NLK filter twice in dual layer. This postprocessing method is then extended to Overcomplete Dual Non-local Kuan’s (OCDNLK) filter by using an overcomplete structure. OCDNLK filter can achieve a better PSNR gain but requires more computation.
Experiments on three test quantization tables
Three quantization tables Q1, Q2, Q3 in Table 4 are widely used to test postprocessing performance, so we first provide the experimental results on images using the three test tables and compare them with the state-of-the-arts methods.
Conclusions
In this paper, a LMMSE Kuan’s filter in the DCT domain is proposed to estimate the original DCT coefficients from quantized DCT coefficients. Because of Non-local parameter estimation technique, the proposed filter is more accurate than the original Kuan’s filter based on local statistics over a uniform window. In addition, we examined the validity of the two assumptions required for the filter and verified each of them. The proposed DCT domain Kuan’s filter for low frequency DCT coefficients
References (33)
- et al.
A smoothness constraint set based on local statistics of BDCT coefficients for image postprocessing
Image Vision Comput.
(2005) - et al.
Reduction of blocking effect in image coding
Opt. Eng.
(1984) - et al.
Nonlinear space-variant postprocessing of block coded images
IEEE Trans. Acoust. Speech Signal Process.
(1986) - et al.
Adaptive deblocking filter
IEEE Trans. Circuits Syst. Video Technol.
(2003) - MPEG4 Verification Model, VM 18.0, 2001. pp....
- et al.
Reduction of block-transform image coding artifacts by using local statistics of transform coefficients
IEEE Signal Process. Lett.
(1997) - et al.
Adaptive postfiltering of transform coefficients for the reduction of blocking artifacts
IEEE Trans. Circuits Syst. Video Technol.
(2001) - et al.
Efficient DCT-domain blind measurement and reduction of blocking artifacts
IEEE Trans. Circuits Syst. Video Technol.
(2002) Iterative procedures for reduction of blocking effects in transform image coding
IEEE Trans. Circuits Syst. Video Technol.
(1992)- et al.
On the POCS-based postprocessing technique to reduce the blocking artifacts in transform coded images
IEEE Trans. Circuits Syst. Video Technol.
(1998)
Projection-based spatially adaptive reconstruction of block-transform compressed images
IEEE Trans. Circuits Syst. Video Technol.
Theory of projection onto the narrow quantization constraint set and its application
IEEE Trans. Image Process.
Artifact reduction in low bit rate DCT-based image compression
IEEE Trans. Image Process.
Reduction of blocking artifacts in image and video coding
IEEE Trans. Circuits Syst. Video Technol.
DCT quantization noise in compressed images
IEEE Trans. Circuits Syst. Video Technol.
Cited by (13)
A novel image deblocking approach within a graph framework
2022, Digital Signal Processing: A Review JournalCitation Excerpt :For instance, Foi et al. developed a novel model for image filtering based on shape-adaptive DCT [7]; Zhai et al. presented an approach through post-filtering in shifted windows of image blocks for image deblocking [8]; They also proposed another deblocking scheme that involves three parts: local ac coefficient regularization of shifted blocks in the DCT domain, block-wise shape adaptive filtering in the spatial domain, and quantization constraint in the DCT domain [9]. Since both nonlocal means and bilateral filters have better effects on image deblocking, several nonlocal filters have been proposed for image deblocking [10–12]. Farinella and Battiato studied the performance of strict sparse coding model selection under lossy JPEG compression, yielding a significant margin of performance improvement [13].
Local statistics and non-local mean filter for speckle noise reduction in medical ultrasound image
2016, NeurocomputingCitation Excerpt :The weighting factors decrease with distance from the pixel of interest and increase when the variance within the window increases. Once the transformation of the multiplicative noise model into a signal-dependent additive noise model is combined with the minimum square error criterion, Kuan filter can generate the linear MMSE for an image that is corrupted with uncorrelated, image-dependent noise [13]. The form of Kuan filter is similar to that of Lee filter, but it uses a different weighting value.
Self-learning-based post-processing for image/video deblocking via sparse representation
2014, Journal of Visual Communication and Image RepresentationCitation Excerpt :To evaluate the proposed deblocking algorithm for compressed images/video, two state-of-the-art image/video deblocking methods [16–17] as well as the built-in H.264/AVC in-loop filter [15] were used for comparisons with our method. The two deblocking methods are: (1) the dual non-local Kuan’s (DNLK) filter based on a DCT domain Kuan’s filter with non-local parameter estimation proposed in [16]; and (2) a signal adaptive weighted sum (SAWS) technique to block boundary pixels proposed in [17]. The deblocking results for the JPEG-decoded Lena image obtained by the DNLK, SAWS, and proposed methods are shown in Fig. 7.
Graph-Based Non-Convex Low-Rank Regularization for Image Compression Artifact Reduction
2020, IEEE Transactions on Image ProcessingAn End-to-End Compression Framework Based on Convolutional Neural Networks
2018, IEEE Transactions on Circuits and Systems for Video Technology