Skip to main content
SpringerLink
Log in

Cyclic label propagation for graph semi-supervised learning

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Graph neural networks (GNNs) have emerged as effective approaches for graph analysis, especially in the scenario of semi-supervised learning. Despite its success, GNN often suffers from over-smoothing and over-fitting problems, which affects its performance on node classification tasks. We analyze that an alternative method, the label propagation algorithm (LPA), avoids the aforementioned problems thus it is a promising choice for graph semi-supervised learning. Nevertheless, the intrinsic limitations of LPA on feature exploitation and relation modeling make propagating labels become less effective. To overcome these limitations, we introduce a novel framework for graph semi-supervised learning termed as Cyclic Label Propagation (CycProp for abbreviation), which integrates GNNs into the process of label propagation in a cyclic and mutually reinforcing manner to exploit the advantages of both GNNs and LPA. In particular, our proposed CycProp updates the node embeddings learned by GNN module with the augmented information by label propagation, while fine-tunes the weighted graph of label propagation with the help of node embedding in turn. After the model converges, reliably predicted labels and informative node embeddings are obtained with the LPA and GNN modules respectively. Extensive experiments on various real-world datasets are conducted, and the experimental results empirically demonstrate that the proposed CycProp model can achieve relatively significant gains over the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Figure 1
Figure 2
Figure 3
Figure 4

Similar content being viewed by others

Notes

  1. http://linqs.cs.umd.edu/projects/projects/lbc

  2. http://socialcomputing.asu.edu/pages/datasets

  3. https://scikit-learn.org/stable/

References

  1. Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. JMLR 7 (Nov), 2399–2434 (2006)

    MathSciNet  MATH  Google Scholar 

  2. Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: COMPSTAT, pp 177–186. Springer (2010)

  3. Bui, T.D., Ravi, S., Ramavajjala, V.: Neural graph learning: Training neural networks using graphs. In: WSDM, pp 64–71. ACM (2018)

  4. Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., Sun, X.: Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: AAAI, pp 3438–3445 (2020)

  5. Duran, A.G., Niepert, M.: Learning graph representations with embedding propagation. In: NIPS, pp 5119–5130 (2017)

  6. Gilmer, J., Schoenholz, S.S., Riley, P. F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: ICML (2017)

  7. Grover, A., Leskovec, J.: node2vec: Scalable feature learning for networks. In: SIGKDD, pp 855–864. ACM (2016)

  8. Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NIPS, pp 1024–1034 (2017)

  9. Huang, X., Li, J., Hu, X.: Accelerated attributed network embedding. In: SDM, pp 633–641. SIAM (2017)

  10. Iscen, A., Tolias, G., Avrithis, Y., Chum, O.: Label propagation for deep semi-supervised learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 5070–5079. Computer Vision Foundation / IEEE. https://doi.org/10.1109/CVPR.2019.00521 (2019)

  11. Jacob, Y., Denoyer, L., Gallinari, P.: Learning latent representations of nodes for classifying in heterogeneous social networks. In: WSDM, pp 373–382. ACM (2014)

  12. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)

  13. Kudo, T., Maeda, E., Matsumoto, Y.: An application of boosting to graph classification. In: NIPS, pp 729–736 (2005)

  14. Kumar, M.P., Packer, B., Koller, D.: Self-paced learning for latent variable models. In: NIPS, pp 1189–1197 (2010)

  15. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  16. Leskovec, J., Lang, K.J., Mahoney, M.: Empirical comparison of algorithms for network community detection. In: WWW, pp 631–640. ACM (2010)

  17. Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. In: NIPS, pp 379–387. MIT Press (2015)

  18. Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

  19. Liang, J., Jacobs, P., Sun, J., Parthasarathy, S.: Semi-supervised embedding in attributed networks with outliers. In: Proceedings of the 2018 SIAM International Conference on Data Mining, pp 153–161. SIAM (2018)

  20. Liao, L., He, X., Zhang, H., Chua, T.S.: Attributed social network embedding. In: arXiv:1705.04969 (2017)

  21. Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. Journal of the American Society for Information Science and Technology 58(7), 1019–1031 (2007)

    Article  Google Scholar 

  22. Liu, L., Zhou, T., Long, G., Jiang, J., Dong, X., Zhang, C.: Isometric propagation network for generalized zero-shot learning. In: International Conference on Learning Representations (ICLR) (2021)

  23. Liu, L., Zhou, T., Long, G., Jiang, J., Zhang, C.: Attribute propagation network for graph zero-shot learning. In: AAAI Conference on Artificial Intelligence (AAAI) (2020)

  24. Lu, Q., Getoor, L.: Link-based classification. In: ICML, pp 496–503 (2003)

  25. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: ICLR (2013)

  26. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp 3111–3119 (2013)

  27. Pan, S., Wu, J., Zhu, X., Zhang, C., Wang, Y.: Tri-party deep network representation. In: IJCAI, pp 1895–1901 (2016)

  28. Parikh, N., Boyd, S.: Proximal algorithms. Foundations and Trends in Optimization 1(3), 127–239 (2014)

    Article  Google Scholar 

  29. Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: Online learning of social representations. In: SIGKDD, pp 701–710. ACM (2014)

  30. Rong, Y., Huang, W., Xu, T., Huang, J.: Dropedge: Towards deep graph convolutional networks on node classification. In: International Conference on Learning Representations (2019)

  31. Shi, W., Rajkumar, R.: Point-gnn: Graph neural network for 3d object detection in a point cloud. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1711–1719 (2020)

  32. Tang, J., Qu, M., Mei, Q.: Pte: Predictive text embedding through large-scale heterogeneous text networks. In: SIGKDD, pp 1165–1174. ACM (2015)

  33. Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: Large-scale information network embedding. In: WWW, pp 1067–1077. ACM (2015)

  34. Tang, L., Liu, H.: Relational learning via latent social dimensions. In: SIGKDD, pp 817–826. ACM (2009)

  35. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: ICLR (2018)

  36. Veličković, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. In: International Conference on Learning Representations (2019)

  37. Wang, B., Tu, Z., Tsotsos, J.K.: Dynamic label propagation for semi-supervised multi-class multi-label classification. In: ICCV, pp. 425–432 (2013)

  38. Wang, F., Zhang, C.: Label propagation through linear neighborhoods. In: ICML, pp 985–992. ACM (2006)

  39. Wang, H., Leskovec, J.: Unifying graph convolutional neural networks and label propagation. arXiv:2002.06755 (2020)

  40. Wang, W., Carreira-Perpinán, M.A.: Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application. In: arXiv:1309.1541 (2013)

  41. Wu, F., Souza, Jr, A.H., Zhang, T., Fifty, C., Yu, T., Weinberger, K.Q.: Simplifying graph convolutional networks. In: ICML (2019)

  42. Wu, L., Wang, D., Feng, S., Zhang, Y., Yu, G.: Mdal: Multi-task dual attention lstm model for semi-supervised network embedding. In: International Conference on Database Systems for Advanced Applications, pp 468–483. Springer (2019)

  43. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems (2020)

  44. Wu, Z., Pan, S., Long, G., Jiang, J., Chang, X., Zhang, C.: Connecting the dots: Multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 753–763 (2020)

  45. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks?. In: International Conference on Learning Representations (2019)

  46. Yang, Z., Cohen, W.W., Salakhutdinov, R.: Revisiting semi-supervised learning with graph embeddings. In: ICML, pp. 40–48. JMLR.org (2016)

  47. Ying, Z., Bourgeois, D., You, J., Zitnik, M., Leskovec, J.: GNNExplainer: Generating explanations for graph neural networks. In: Advances in neural information processing systems, pp. 9244–9255 (2019)

  48. Zhou, D., Bousquet, O., Lal, T. N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: NIPS, pp. 321–328 (2004)

  49. Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using Gaussian fields and harmonic functions. In: ICML, pp. 912–919 (2003)

Download references

Author information

Authors and Affiliations

  1. Alibaba Group, Hangzhou, China

    Zhao Li

  2. Monash University, Clayton, Australia

    Yixin Liu & Shirui Pan

  3. Zhejiang University, Hangzhou, China

    Zhen Zhang & Jiajun Bu

  4. Central South University, Changsha, China

    Jianliang Gao

Authors
  1. Zhao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yixin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shirui Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianliang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiajun Bu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhao Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Large Scale Graph Data Analytics

Guest Editors: Xuemin Lin, Lu Qin, Wenjie Zhang, and Ying Zhang

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Z., Liu, Y., Zhang, Z. et al. Cyclic label propagation for graph semi-supervised learning. World Wide Web 25, 703–721 (2022). https://doi.org/10.1007/s11280-021-00906-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-021-00906-2

Keywords

Search

Navigation