Image postprocessing by Non-local Kuan’s filter

https://doi.org/10.1016/j.jvcir.2010.12.007Get rights and content

Abstract

Blocking artifacts exist in images and video sequences compressed to low bit rates using block-based discrete cosine transform (DCT) compression standards. In order to reduce blocking artifacts, two image postprocessing techniques, DNLK filter and OCDNLK filter, are presented in this paper. A more accurate DCT domain Kuan’s filter based on Non-local parameter estimation was proposed from the linear minimum mean-square-error (MMSE) criterion. We analyze the required two assumptions for the filter theoretically. Then the DCT domain Kuan’s filter for low frequency coefficients and Non-local mean filter for high frequency AC coefficients constitute the proposed Non-local Kuan’s (NLK) filter. After that, we propose the Dual Non-local Kuan’s (DNLK) filter by applying the proposed filter in dual layer. The DNLK filter is extended to form the Overcomplete Dual Non-local Kuan’s (OCDNLK) filter by applying to the overcomplete DCT coefficients. Experimental results on coded images using test quantization tables and JPEG coded images show the effectiveness of the two methods.

Research highlights

► A DCT domain Non-local Kuan’s filter is proposed. ► The required assumptions for the filter are analyzed theoretically. ► Dual layer filtering process is used to restore DCT coefficients. ► Blocking artifacts of coded images are efficiently removed.

Introduction

Block-based discrete cosine transform (DCT) coding is adopted in several industry standards, such as JPEG, MPEG, and H.264, for image and video compression. In a block-based DCT scheme, an image is firstly divided into non-overlapping blocks and pixels in each block are transformed into the DCT coefficients which are then quantized. At low bit rates, blocking artifacts will be visible, because quantization is performed independently at each block without considering the existing correlations among adjacent blocks. The use of a postprocessing technique on a coded image is a common strategy to reduce blocking artifacts and improve fidelity, as it does not require modifications of existing image/video standards and so is readily available. Based on different types of postprocessing techniques, such as post-filtering, iterated projection onto convex sets (POCS), maximum a posteriori (MAP) estimation, overcomplete wavelet, a variety of postprocessing methods have been developed to cope with the problem.

Early attempts used image enhancement approach. For example, Lim and Reeve [1] applied low-pass filtering to pixels along block boundaries. This method sometimes blurs true edges, so some adaptive post-filtering techniques were proposed. Ramamurthi and Gersho [2] applied 2D low-pass filtering only to areas away from edges and performed 1D filtering to areas near edges to avoid blurring them. The post-filtering techniques used in recent coding standards such as H.264/AVC [3] and MPEG-4 [4] consist of several filters and one of them is selected based on local activity. The abovementioned methods are performed in the spatial domain. Recently, some post-filtering techniques in the DCT domain were proposed [5], [6], [7], which have demonstrated that postprocessing in transform domain is a promising postprocessing approach.

Methods based on POCS techniques [8], [9], [10], [11], [12] were also proposed. The idea is to impose smoothness constraint around block boundaries and project block images onto these convex sets (POCS) iteratively. Both forward and inverse DCT are required in each iteration, so the major drawback is high computational complexity. In order to reduce computation, Gan et al. [12] proposed a novel smoothness constraint set in the DCT domain and used the iterative POCS technique to reduce blocking artifacts.

By formulating postprocessing as an inverse problem, the MAP estimation technique can be used to find the solution. The probability function of the original image is modeled by different Markov random field (MRF) models [13], [14], [15], [16], [17]. Sun and Cham [16], [17] modeled the original image as a high order MRF using the Fields of Experts (FoE) framework, and proposed an effective postprocessing method by MAP criterion. Li and Delp [18] addressed the problem using a DCT domain MRF model to do the MAP estimation.

As to the overcomplete wavelet technique [19], [20], [21], it uses the overcomplete wavelet representation to reduce blocking artifacts. Xiong et al. [19] used thresholding of the overcomplete wavelet coefficients to reduce the quantization effects. In [20] the wavelet transform modulus maxima representation was adopted for deblocking. Liew and Yan [21] analyzed the block discontinuities caused by coding to derive more accurate thresholds at different wavelet scales. Recently, a novel deblocking method based on the shape-adaptive DCT [22] was developed, which achieved the best performance until now.

Kuan’s filter [24] can produce the linear minimum mean-square-error (MMSE) for a signal corrupted with uncorrelated, signal-dependent noise. We develop a more accurate DCT domain Kuan’s filter using Non-local parameter estimation technique. Two assumptions are needed to derive the solution of the filter. We investigate the validity of the assumptions and verify each of them. The DCT domain Kuan’s filter is applied for low frequency DCT coefficients that satisfy the two assumptions and Non-local mean filter is used for high frequency AC coefficients. Then we have the proposed Non-local Kuan’s (NLK) filter. A new image postprocessing method called Dual Non-local Kuan’s (DNLK) filter is then proposed. It uses the NLK filter to obtain good estimates of image statistics and then applies the NLK filter again to reduce quantization noise. We also found that the proposed NLK filter can be used for the estimation of the overcomplete DCT coefficients. Hence, the DNLK filter is combined with the overcomplete representation to form the Overcomplete Dual Non-local Kuan’s (OCDNLK) filter. Experimental results show that the OCDNLK filter, in most cases, can achieve higher PSNR gain than other state-of-the-arts methods and generates images with the best visual quality. The performance of the DNLK filter, which has lower complexity than the OCDNLK filter, is comparable to other state-of-the-arts methods. Moreover, we demonstrate the efficiency of the two methods on JPEG coded images under various image quality settings.

In Section 2, we explain the proposed Kuan’s filter in the DCT domain using Non-local parameter estimation technique which achieves LMMSE, analyze the two assumptions required, and finally develop the NLK filter used for all DCT coefficients. Section 3 presents the two image postprocessing methods, DNLK filter and OCDNLK filter. Experimental results on images using test quantization tables and JPEG coded images are given in Section 4. Finally we draw conclusions in Section 5.

Section snippets

The proposed Non-local Kuan’s filter in the DCT domain

A roundoff quantizer is a non-linear device having a staircase-type input–output characteristic as shown in Fig. 1. At the transmitting end, if we know the original DCT coefficients x0 and quantization step Q, the quantized value y0 ϵ {rQ ; r = 0, ±1, ±2, …} and quantization error n will be determined. However, at the receiving end, we have the quantized value y0, and only know that error n is in the range [−Q/2, Q/2] and the input x0 is in the range [y0  Q/2, y0 + Q/2]. Let y0 = x0 + n, where n is the

Image postprocessing by the dual Non-local Kuan’s filter

In Section 2, we have described the proposed NLK filter and methods for estimation of its parameters. In this section, we first propose a postprocessing method called Dual Non-local Kuan’s (DNLK) filter by applying this NLK filter twice in dual layer. This postprocessing method is then extended to Overcomplete Dual Non-local Kuan’s (OCDNLK) filter by using an overcomplete structure. OCDNLK filter can achieve a better PSNR gain but requires more computation.

Experiments on three test quantization tables

Three quantization tables Q1, Q2, Q3 in Table 4 are widely used to test postprocessing performance, so we first provide the experimental results on images using the three test tables and compare them with the state-of-the-arts methods.

Conclusions

In this paper, a LMMSE Kuan’s filter in the DCT domain is proposed to estimate the original DCT coefficients from quantized DCT coefficients. Because of Non-local parameter estimation technique, the proposed filter is more accurate than the original Kuan’s filter based on local statistics over a uniform window. In addition, we examined the validity of the two assumptions required for the filter and verified each of them. The proposed DCT domain Kuan’s filter for low frequency DCT coefficients

References (33)

  • Xiangchao Gan et al.

    A smoothness constraint set based on local statistics of BDCT coefficients for image postprocessing

    Image Vision Comput.

    (2005)
  • J.S. Lim et al.

    Reduction of blocking effect in image coding

    Opt. Eng.

    (1984)
  • B. Ramamurthi et al.

    Nonlinear space-variant postprocessing of block coded images

    IEEE Trans. Acoust. Speech Signal Process.

    (1986)
  • P. List et al.

    Adaptive deblocking filter

    IEEE Trans. Circuits Syst. Video Technol.

    (2003)
  • MPEG4 Verification Model, VM 18.0, 2001. pp....
  • Steven S.O. Choy et al.

    Reduction of block-transform image coding artifacts by using local statistics of transform coefficients

    IEEE Signal Process. Lett.

    (1997)
  • Tao Chen et al.

    Adaptive postfiltering of transform coefficients for the reduction of blocking artifacts

    IEEE Trans. Circuits Syst. Video Technol.

    (2001)
  • Shizhong Liu et al.

    Efficient DCT-domain blind measurement and reduction of blocking artifacts

    IEEE Trans. Circuits Syst. Video Technol.

    (2002)
  • A. Zakhor

    Iterative procedures for reduction of blocking effects in transform image coding

    IEEE Trans. Circuits Syst. Video Technol.

    (1992)
  • H. Paek et al.

    On the POCS-based postprocessing technique to reduce the blocking artifacts in transform coded images

    IEEE Trans. Circuits Syst. Video Technol.

    (1998)
  • Y. Yang et al.

    Projection-based spatially adaptive reconstruction of block-transform compressed images

    IEEE Trans. Circuits Syst. Video Technol.

    (1995)
  • S.H. Park et al.

    Theory of projection onto the narrow quantization constraint set and its application

    IEEE Trans. Image Process.

    (1999)
  • J. Luo et al.

    Artifact reduction in low bit rate DCT-based image compression

    IEEE Trans. Image Process.

    (1996)
  • T. Meier et al.

    Reduction of blocking artifacts in image and video coding

    IEEE Trans. Circuits Syst. Video Technol.

    (1999)
  • M. Robertson et al.

    DCT quantization noise in compressed images

    IEEE Trans. Circuits Syst. Video Technol.

    (2005)
  • D. Sun, W.-K. Cham, An effective postprocessing method for low bit rate block DCT coded images, in: IEEE Int. Conf....
  • Cited by (13)

    • A novel image deblocking approach within a graph framework

      2022, Digital Signal Processing: A Review Journal
      Citation Excerpt :

      For instance, Foi et al. developed a novel model for image filtering based on shape-adaptive DCT [7]; Zhai et al. presented an approach through post-filtering in shifted windows of image blocks for image deblocking [8]; They also proposed another deblocking scheme that involves three parts: local ac coefficient regularization of shifted blocks in the DCT domain, block-wise shape adaptive filtering in the spatial domain, and quantization constraint in the DCT domain [9]. Since both nonlocal means and bilateral filters have better effects on image deblocking, several nonlocal filters have been proposed for image deblocking [10–12]. Farinella and Battiato studied the performance of strict sparse coding model selection under lossy JPEG compression, yielding a significant margin of performance improvement [13].

    • Local statistics and non-local mean filter for speckle noise reduction in medical ultrasound image

      2016, Neurocomputing
      Citation Excerpt :

      The weighting factors decrease with distance from the pixel of interest and increase when the variance within the window increases. Once the transformation of the multiplicative noise model into a signal-dependent additive noise model is combined with the minimum square error criterion, Kuan filter can generate the linear MMSE for an image that is corrupted with uncorrelated, image-dependent noise [13]. The form of Kuan filter is similar to that of Lee filter, but it uses a different weighting value.

    • Self-learning-based post-processing for image/video deblocking via sparse representation

      2014, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      To evaluate the proposed deblocking algorithm for compressed images/video, two state-of-the-art image/video deblocking methods [16–17] as well as the built-in H.264/AVC in-loop filter [15] were used for comparisons with our method. The two deblocking methods are: (1) the dual non-local Kuan’s (DNLK) filter based on a DCT domain Kuan’s filter with non-local parameter estimation proposed in [16]; and (2) a signal adaptive weighted sum (SAWS) technique to block boundary pixels proposed in [17]. The deblocking results for the JPEG-decoded Lena image obtained by the DNLK, SAWS, and proposed methods are shown in Fig. 7.

    • An End-to-End Compression Framework Based on Convolutional Neural Networks

      2018, IEEE Transactions on Circuits and Systems for Video Technology
    View all citing articles on Scopus
    View full text