Neighborhood-consensus message passing as a framework for generalized iterated conditional expectations
Highlights
► Inference method for maximum a posteriori estimation in Markov random field model. ► Neighborhood-based message passing framework. ► Combination of iterated conditional modes and loopy belief propagation. ► Proposing also simplified version called weighted iterated conditional modes ► Method is simple and performs better in terms of speed and/or quality than related techniques.
Introduction
A typical problem in image processing consists of estimating some unknown image attributes from the available image data, which are incomplete or degraded. The unknown attributes can be the noise free components of the noisy image pixels, values of disparities from a stereo pair, missing pixel values, segments of the image to which each pixel belongs etc. This problem is usually referred to as pixel-labeling: each pixel is assigned a label representing the desired attribute. Pixel-labeling usually involves Bayesian inference like maximum a posteriori (MAP) estimation with a Markov Random Field (MRF) prior Besag, 1986, Li, 1995 and it is also referred to as an energy minimization problem.
In these MAP-MRF labeling problems computation is typically exhaustive or even intractable due to a large number of variables and loopy structure of the graph. Classical inference algorithms include Monte Carlo Markov Chain (MCMC) samplers, such as Gibbs and Metropolis sampler Li (1995), which are slow but find an optimal solution with high probability. A popular suboptimal algorithm called iterated conditional modes (ICM) Besag (1986) is a “greedy” method that reaches only a local optimum. More recent techniques involve graph cuts (GC) Boykov et al., 2001, Greig et al., 1989 and message passing algorithms such as loopy belief propagation (LBP) Pearl, 1988, Yedidia and Freeman, 2001 and tree-reweighted message passing Kolmogorov (2006). Graph cuts give optimal solution for binary MRFs and very good result for multi-label MRFs but with the limitation of being applicable without modifications to only certain class of problems Kolmogorov and Zabih (2004). An excellent overview and comparison of these and other inference methods is in (Szeliski et al., 2008).
Although LBP gives state-of-the-art results in the fields of error-correcting codes Frey and MacKay (1998) and computer vision (stereo matching, super-resolution Freeman et al. (2000), etc.), it has been reported to fail for graphs with huge number of nodes and many short loops Murphy et al. (1999). Moreover, LBP tends to be slow on these types of graphs, which appear in many practical image processing problems. In our experiments, LBP was less efficient than MCMC samplers and even less efficient than the greedy ICM in inferring the spatial clustering of sparse image coefficients (such as wavelet or shearlet Easley et al. (2008) coefficients). Inferring the spatial structure in sparse image representations is crucial for emerging model-based compressed sensing Baraniuk et al. (2010) and structured sparsity approaches He and Carin, 2009, Huang et al., 2009 in general. Triggered by these problems, we propose a novel suboptimal inference algorithm, which performs well on huge graphs with many short loops and which allows great flexibility in defining spatial interactions between the nodes. These are exactly the cases where LBP often fails or becomes impractical. The proposed scheme has the computational simplicity and robustness similar to ICM and the flexibility of a more general message passing.
The central idea to our approach is to propagate information through the graph by sending a single “consensus” message from the neighborhood to the central node. Therefore, we name the proposed method neighborhood-consensus message passing (NCMP). Similarly to ICM, the message represents a unified opinion of the whole neighborhood about the labels of the central node. Contrasting to ICM, we also take into account additional information in the form of probabilities of all neighboring labels that form their “voting” for the labels of the central node. Hence, the message is a function of beliefs of the neighboring nodes representing confidence about their own labels. The proposed approach can also be considered as a generalization of iterated conditional expectations (ICE) Owen (1989) within a message passing framework. The ICE algorithm was developed as an extension of ICM, but despite its great potentials, it is being neglected in the recent literature. We revisit this idea here and redefine it in a message passing framework which makes it suitable for generalizations and extensions. Furthermore, we develop another version of our NCMP based message passing that we call weighted iterated conditional modes (WICM) because it reminds of ICM with additional weighting of the neighboring nodes’ influence.
Experiments show that the proposed NCMP approach outperforms ICM, always reaching better optimum, but at the expense of taking more iterations to converge. In comparison with LBP, NCMP is computationally simpler and faster, while it usually yields comparable results in terms of correct labeling. This performance comparison is application dependent. The results demonstrate a clear improvement over LBP in inferring the structure of sparse image coefficients. In this type of problems there is a clear benefit from using the proposed approach: it is faster and yields better result than LBP. The convergence plots show that LBP and NCMP reach nearly the same optimum. LBP takes usually fewer iterations to converge, but each of these iterations lasts longer, so the convergence time of LBP is usually longer than that of NCMP. Moreover, our method is directly applicable to different MRF models, e.g. hierarchical or non-submodular, which is an advantage in comparison with GC. ICM has the same flexibility, but its performance is limited because it easily gets trapped in the local optimum. Finally, our method is fast and simple to implement since it is based on local computations. Plausible result is achieved in only a few iterations. We stress that the proposed NCMP framework remains inferior to the more sophisticated LBP in more demanding applications, but achieves better performance in terms of quality and/or speed for certain problems, especially when they include large graphs with high connectivity. The proposed method is definitely an interesting alternative to other low-complexity methods like ICM.
The paper is organized as follows. In Section 2 we set the theoretical background by defining MRFs. We also review briefly ICM, LBP and ICE as reference algorithms. Section 3 introduces a definition of NCMP and a novel WICM as its special case. Example applications and performance comparison are given in Section 4. Finally, we conclude the paper with Section 5.
Section snippets
Markov random fields
A general image model that we consider is sketched in Fig. 1. In this representation, observed nodes represent given image data y, e.g. image pixels or patches. Each observed node is connected to the corresponding hidden node. A set of hidden nodes is denoted as vector x, the elements of which can take one of L values (usually referred to as labels). Thus, xi denotes a label of node i, where . xA denotes a set of nodes with indices in the set A. The connections between the hidden
Neighborhood-consensus message passing
Our idea is to simplify LBP algorithm and make it better suited for networks with huge number of nodes and short loops, which are the cases in which LBP was reported to fail Murphy et al. (1999). In this situation, messages are unable to convey the necessary information globally throughout the graph. The solution can be to observe a larger neighborhood but then the speed and the practicality become an issue. We believe that these problems are caused by the message being defined as a pairwise
Experiments and results
In this section we present a few example applications that illustrate the potentials of the proposed approach. We consider both binary and multi-label MRFs with second and first-order neighborhoods. We compare the proposed approach with the reference methods, namely ICM, LBP and GC, on the same examples and the same set of parameters. For comparison of binary denoising example from Section 4.1 (except for LBP because the code does not support second-order neighborhood) and binary segmentation
Conclusion and future work
In this paper, we propose a new inference method which generalizes iterated conditional expectations within a message passing framework. Practically, this approach is a simple modification of ICM but formulated as message passing similar to that in LBP. We call this method neighborhood-consensus message passing (NCMP) since a joint message is sent from the specified neighborhood to the central node which enables information to propagate through the graph. Information consists of beliefs of
References (26)
- et al.
Sparse directional image representations using the discrete shearlet transform
Appl. Comput. Harmon. Anal.
(2008) - et al.
Model-based compressive sensing
IEEE Trans. Inform. Theory
(2010) On the statistical analysis of dirty pictures (with discussion)
J. Roy. Stat. Soc. Ser. B
(1986)Pattern Recognition and Machine Learning
(2006)- et al.
Fast approximate energy minimization via graph cuts
IEEE Trans. Pattern Anal. Mach. Intell.
(2001) - et al.
Learning low-level vision
Int. J. Comput. Vision
(2000) - et al.
A revolution: Belief propagation in graphs with cycles
- et al.
Exact maximum a posteriori estimation for binary images
J. Roy. Stat. Soc. Ser. B (Methodological)
(1989) - et al.
Exploiting structure in wavelet-based Bayesian compressive sensing
IEEE Trans. Sig. Proc.
(2009) - et al.
Learning with structured sparsity
Convergent tree-reweighted message passing for energy minimization
IEEE Trans. Pattern Anal. Mach. Intell.
What energy functions can be minimized via graph cuts?
IEEE Trans. Pattern Anal. Mach. Intell.
Markov Random Field Modeling in Computer Vision
Cited by (7)
Neighborhood Feature Extraction and Haralick Attributes for Medical Image Analysis: Application to Breast Cancer Mammography Image
2023, Proceedings - 10th International Conference on Wireless Networks and Mobile Communications, WINCOM 2023Fractal analysis segmentation of medical images: Application to breast cancer mammography image
2022, Proceedings - 2022 5th International Conference on Advanced Communication Technologies and Networking, CommNet 2022From textural inpainting to deep generative models: An extensive survey of image inpainting techniques
2020, Journal of Computer ScienceContext-aware patch-based image inpainting using Markov random field modeling
2015, IEEE Transactions on Image ProcessingTotal Variation denoising using iterated conditional expectation
2014, European Signal Processing ConferenceDepth-guided patch-based disocclusion filling for view synthesis via Markov random field modelling
2014, 2014, 8th International Conference on Signal Processing and Communication Systems, ICSPCS 2014 - Proceedings
- 1
A. Pižurica is a post-doctoral researcher of the Fund for the Scientific Research in Flanders, FWO.