Skip to main content
SpringerLink
Log in

Exhaustive and Efficient Constraint Propagation: A Graph-Based Learning Approach and Its Applications

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper presents a novel pairwise constraint propagation approach by decomposing the challenging constraint propagation problem into a set of independent semi-supervised classification subproblems which can be solved in quadratic time using label propagation based on \(k\)-nearest neighbor graphs. Considering that this time cost is proportional to the number of all possible pairwise constraints, our approach actually provides an efficient solution for exhaustively propagating pairwise constraints throughout the entire dataset. The resulting exhaustive set of propagated pairwise constraints are further used to adjust the similarity matrix for constrained spectral clustering. Other than the traditional constraint propagation on single-source data, our approach is also extended to more challenging constraint propagation on multi-source data where each pairwise constraint is defined over a pair of data points from different sources. This multi-source constraint propagation has an important application to cross-modal multimedia retrieval. Extensive results have shown the superior performance of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Bartels, R., & Stewart, G. (1972). Solution of the matrix equation \({AX+XB=C}\). Communications of the ACM, 15(9), 820–826.

    Article  Google Scholar 

  • Basu, S., Bilenko, M., & Mooney, R. (2004). A probabilistic framework for semi-supervised clustering. In Proceedings of the SIGKDD (pp. 59–68).

  • Bekkerman, R., & Jeon, J. (2007). Multi-modal clustering for multimedia collections. In Proceedings of the CVPR (pp. 1–8).

  • Blei, D., Ng, A., & Jordan, M. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.

    MATH  Google Scholar 

  • Bruno, E., Moenne-Loccoz, N., & Marchand-Maillet, S. (2008). Design of multimodal dissimilarity spaces for retrieval of video documents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(9), 1520–1533.

    Article  Google Scholar 

  • Carson, C., Belongie, S., Greenspan, H., & Malik, J. (2002). Blobworld: Image segmentation using expectation-maximization and its application to image querying. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(8), 1026–1038.

    Article  Google Scholar 

  • Chen, G., & Lerman, G. (2009). Spectral curvature clustering (scc). International Journal of Computer Vision, 81, 317–330.

    Article  Google Scholar 

  • Eaton, E., desJardins, M., & Jacob, S. (2010). Multi-view clustering with constraint propagation for learning with an incomplete mapping between views. In Proceedings of the ACM conference on information and knowledge management (CIKM) (pp. 389–398).

  • Eriksson, A., Olsson, C., & Kahl, F. (2007). Normalized cuts revisited: A reformulation for segmentation with linear grouping constraints. In Proceedings of the ICCV (pp. 1–8).

  • Farhadi, A., Hejrati, M., Sadeghi, M., Young, P., Rashtchian, C., Hockenmaier, J., et al. (2010). Every picture tells a story: Generating sentences from images. In Proceedings of the ECCV (Vol. 4, pp. 15–29).

  • Feng, S., Manmatha, R., & Lavrenko, V. (2004). Multiple Bernoulli relevance models for image and video annotation. In Proceedings of the CVPR (Vol. 2, pp. 1002–1009).

  • Gajic, Z., & Qureshi, M. (Eds.). (1995). Lyapunov matrix equation in system stability and control. New York: Academic Press.

  • Ghanem, B., & Ahuja, N. (2010). Dinkelbach NCUT: An efficient framework for solving normalized cuts problems with priors and convex constraints. International Journal of Computer Vision, 89(1), 40–55.

    Article  Google Scholar 

  • Hoi, S., Liu, W., Lyu, M., & Ma, W. Y. (2006). Learning distance metrics with contextual constraints for image retrieval. In Proceedings of the CVPR (pp. 2072–2078).

  • Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28(3–4), 321–377.

    MATH  Google Scholar 

  • Hubert, L., & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2(1), 193–218.

    Article  Google Scholar 

  • Kamvar, S., Klein, D., & Manning, C. (2003). Spectral learning. In Proceedings of the IJCAI (pp. 561–566).

  • Klein, D., Kamvar, S., & Manning, C. (2002). From instance-level constraints to space-level constraints: Making the most of prior knowledge in data clustering. In Proceedings of the ICML (pp. 307–314).

  • Kulis, B., Basu, S., Dhillon, I., & Mooney, R. (2005). Semi-supervised graph clustering: A kernel approach. In Proceedings of the ICML (pp. 457–464)..

  • Kulkarni, G., Premraj, V., Dhar, S., Li, S., Choi, Y., Berg, A., et al. (2011). Baby talk: Understanding and generating simple image descriptions. In Proceedings of the CVPR (pp. 1601–1608).

  • Lancaster, P. (1970). Explicit solutions of linear matrix equations. SIAM Review, 12(4), 544–566.

    Article  MathSciNet  MATH  Google Scholar 

  • Law, M., Topchy, A., & Jain, A. (2004). Clustering with soft and group constraints. In Proceedings of the joint IAPR international workshop on structural, syntactic, and statistical pattern recognition (pp. 662–670).

  • Li, Z., Liu, J., & Tang, X. (2008). Pairwise constraint propagation by semidefinite programming for semi-supervised classification. In Proceedings of the ICML (pp. 576–583).

  • Li, J., & Wang, J. (2003). Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1075–1088.

    Article  Google Scholar 

  • Liu, W., Ma, S., Tao, D., Liu, J., & Liu, P. (2010). Semi-supervised sparse metric learning using alternating linearization optimization. In Proceedings of the KDD (pp. 1139–1148).

  • Lu, Z., & Carreira-Perpinan, M. (2008). Constrained spectral clustering through affinity propagation. In Proceedings of the CVPR.

  • Lu, Z., & Ip, H. (2009). Image categorization by learning with context and consistency. In Proceedings of the CVPR (pp. 2719–2726).

  • Lu, Z., & Ip, H. (2010). Constrained spectral clustering via exhaustive and efficient constraint propagation. In Proceedings of the ECCV (Vol, 6, pp. 1–14).

  • Lu, Z., Ip, H., & Peng, Y. (2011). Contextual kernel and spectral methods for learning the semantics of images. IEEE Transactions on Image Processing, 20(6), 1739–1750.

    Article  MathSciNet  Google Scholar 

  • Lu, Z., Peng, Y., & Xiao, J. (2008). From comparing clusterings to combining clusterings. In Proceedings of the AAAI (pp. 665–670).

  • Ng, A., Jordan, M., & Weiss, Y. (2002). On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems, 14, 849–856.

    Google Scholar 

  • Oliva, A., & Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42(3), 145–175.

    Article  MATH  Google Scholar 

  • Ordonez, V., Kulkarni, G., & Berg, T. (2012). Im2Text: Describing images using 1 million captioned photographs. Advances in Neural Information Processing Systems, 24, 1143–1151.

    Google Scholar 

  • Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G., Levy, R., et al. (2010). A new approach to cross-modal multimedia retrieval. In Proceedings of the ACM multimedia (pp. 251–260).

  • Rasiwasia, N., Moreno, P., & Vasconcelos, N. (2007). Bridging the gap: Query by semantic example. IEEE Transactions on Multimedia, 9(5), 923–938.

    Article  Google Scholar 

  • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.

    Article  Google Scholar 

  • Snoek, C., & Worring, M. (2005). Multimodal video indexing: A review of the state-of-the-art. Multimedia Tools and Applications, 25(1), 5–35.

    Article  Google Scholar 

  • Veksler, O. (2008). Star shape prior for graph-cut image segmentation. In Proceedings of the ECCV (pp. 454–467).

  • von Luxburg, U. (2007). A tutorial on spectral clustering. Statistics and Computing, 17(4), 395–416.

    Article  MathSciNet  Google Scholar 

  • Wagstaff, K., Cardie, C., Rogers, S., & Schroedl, S. (2001). Constrained k-means clustering with background knowledge. In Proceedings of the ICML (pp. 577–584).

  • Xing, E., Ng, A., Jordan, M., & Russell, S. (2003). Distance metric learning with application to clustering with side-information. Advances in Neural Information Processing Systems, 15, 505–512.

    Google Scholar 

  • Xu, L., Li, W., & Schuurmans, D. (2009). Fast normalized cut with linear constraints. In Proceedings of the CVPR (pp. 2866–2873).

  • Yu, S., & Shi, J. (2004). Segmentation given partial grouping constraints. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2), 173–183.

    Article  Google Scholar 

  • Zhou, D., Bousquet, O., Lal, T., Weston, J., & Schölkopf, B. (2004). Learning with local and global consistency. Advances in Neural Information Processing Systems, 16, 321–328.

    Google Scholar 

  • Zhu, X., Ghahramani, Z., & Lafferty, J. (2003). Semi-supervised learning using Gaussian fields and harmonic functions. In Proceedings of the ICML (pp. 912–919).

Download references

Acknowledgments

The authors would like to thank Liwei Wang, Zhenyong Fu, and the anonymous reviewers for their valuable comments. This work was supported by National Natural Science Foundation of China under Grants 61073084 and 61202231, Beijing Natural Science Foundation of China under Grant 4122035, National Hi-Tech Research and Development Program (863 Program) of China under Grant 2012AA012503, National Development and Reform Commission High-tech Program of China under Grant [2010]3044, and National Key Technology Research and Development Program of China under Grant 2012BAH07B01.

Author information

Authors and Affiliations

  1. Institute of Computer Science and Technology, Peking University, Beijing, 100871, China

    Zhiwu Lu & Yuxin Peng

Authors
  1. Zhiwu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuxin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuxin Peng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lu, Z., Peng, Y. Exhaustive and Efficient Constraint Propagation: A Graph-Based Learning Approach and Its Applications. Int J Comput Vis 103, 306–325 (2013). https://doi.org/10.1007/s11263-012-0602-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-012-0602-z

Keywords

Search

Navigation