Xingcheng Fu
ABSTRACT
Learning unbiased node representations for imbalanced samples in the graph has become a more remarkable and important topic. For the graph, a significant challenge is that the topological properties of the nodes (e.g., locations, roles) are unbalanced (topology-imbalance), other than the number of training labeled nodes (quantity-imbalance). Existing studies on topology-imbalance focus on the location or the local neighborhood structure of nodes, ignoring the global underlying hierarchical properties of the graph, i.e., hierarchy. In the real-world scenario, the hierarchical structure of graph data reveals important topological properties of graphs and is relevant to a wide range of applications. We find that training labeled nodes with different hierarchical properties have a significant impact on the node classification tasks and confirm it in our experiments. It is well known that hyperbolic geometry has a unique advantage in representing the hierarchical structure of graphs. Therefore, we attempt to explore the hierarchy-imbalance issue for node classification of graph neural networks with a novelty perspective of hyperbolic geometry, including its characteristics and causes. Then, we propose a novel hyperbolic geometric hierarchy-imbalance learning framework, named HyperIMBA, to alleviate the hierarchy-imbalance issue caused by uneven hierarchy-levels and cross-hierarchy connectivity patterns of labeled nodes. Extensive experimental results demonstrate the superior effectiveness of HyperIMBA for hierarchy-imbalance node classification tasks.
- Réka Albert and Albert-László Barabási. 2000. Topology of evolving networks: local events and universality. Physical review letters 85, 24 (2000), 5234.Google Scholar
- Réka Albert, Hawoong Jeong, and Albert-László Barabási. 1999. Diameter of the world-wide web. nature 401, 6749 (1999), 130–131.Google Scholar
- Albert-László Barabási and Réka Albert. 1999. Emergence of scaling in random networks. Science 286, 5439 (1999), 509–512.Google Scholar
- Silvere Bonnabel. 2013. Stochastic gradient descent on Riemannian manifolds. IEEE Trans. Automat. Control 58, 9 (2013), 2217–2229.Google ScholarCross Ref
- Eliav Buchnik and Edith Cohen. 2018. Bootstrapped graph diffusions: Exposing the power of nonlinearity. In SIGMETRICS. 8–10.Google Scholar
- James W Cannon, William J Floyd, Richard Kenyon, Walter R Parry, 1997. Hyperbolic geometry. Flavors of geometry 31 (1997), 59–115.Google Scholar
- Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. 2019. Learning imbalanced datasets with label-distribution-aware margin loss. In NeurIPS, Vol. 32.Google Scholar
- Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. 2019. Hyperbolic Graph Convolutional Neural Networks. In NeurIPS. 4869–4880.Google Scholar
- Deli Chen, Yankai Lin, Guangxiang Zhao, Xuancheng Ren, Peng Li, Jie Zhou, and Xu Sun. 2021. Topology-Imbalance Learning for Semi-Supervised Node Classification. NeurIPS 34 (2021).Google Scholar
- Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. 2019. Class-balanced loss based on effective number of samples. In CVPR. 9268–9277.Google Scholar
- Sergey N Dorogovtsev, Alexander V Goltsev, and José Ferreira F Mendes. 2002. Pseudofractal scale-free web. Physical review E 65, 6 (2002), 066122.Google Scholar
- Xingcheng Fu, Jianxin Li, Jia Wu, Qingyun Sun, Cheng Ji, Senzhang Wang, Jiajun Tan, Hao Peng, and S Yu Philip. 2021. ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network. In ICDM. IEEE, 111–120.Google Scholar
- Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. 2018. Hyperbolic Neural Networks. In NeurIPS. 5350–5360.Google Scholar
- Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In ICML. 1263–1272.Google Scholar
- Guo Haixiang, Li Yijing, Jennifer Shang, Gu Mingyun, Huang Yuanyue, and Gong Bing. 2017. Learning from class-imbalanced data: Review of methods and applications. Expert systems with applications 73 (2017), 220–239.Google Scholar
- Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1024–1034.Google Scholar
- Haibo He and Edwardo A Garcia. 2009. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering 21, 9 (2009), 1263–1284.Google ScholarDigital Library
- W Sean Kennedy, Iraj Saniee, and Onuttom Narayan. 2016. On the hyperbolicity of large-scale networks and its estimation. In Big Data. IEEE, 3344–3351.Google Scholar
- Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.Google Scholar
- Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguná. 2010. Hyperbolic geometry of complex networks. Physical Review E 82, 3 (2010), 036106.Google ScholarCross Ref
- Jianxin Li, Xingcheng Fu, Qingyun Sun, Cheng Ji, Jiajun Tan, Jia Wu, and Hao Peng. 2022. Curvature Graph Generative Adversarial Networks. In WWW. 1528–1537.Google Scholar
- Jianxin Li, Qingyun Sun, Hao Peng, Beining Yang, Jia Wu, and S Yu Phillp. 2023. Adaptive Subgraph Neural Network with Reinforced Critical Structure Mining. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).Google ScholarDigital Library
- Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In ECCV. 2980–2988.Google Scholar
- Vivi Nastase, Rada Mihalcea, and Dragomir R Radev. 2015. A survey of graphs in natural language processing. Natural Language Engineering 21, 5 (2015), 665–698.Google ScholarCross Ref
- Maximilian Nickel and Douwe Kiela. 2017. Poincaré Embeddings for Learning Hierarchical Representations. In NeurIPS. 6338–6347.Google Scholar
- Yann Ollivier. 2009. Ricci curvature of Markov chains on metric spaces. Journal of Functional Analysis 256, 3 (2009), 810–864.Google ScholarCross Ref
- Fragkiskos Papadopoulos, Maksim Kitsak, M Ángeles Serrano, Marián Boguná, and Dmitri Krioukov. 2012. Popularity versus similarity in growing networks. Nature (2012), 537–540.Google Scholar
- Joonhyung Park, Jaeyun Song, and Eunho Yang. 2021. GraphENS: Neighbor-Aware Ego Network Synthesis for Class-Imbalanced Node Classification. In ICLR.Google Scholar
- Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2020. Geom-GCN: Geometric Graph Convolutional Networks. In ICLR. OpenReview.net.Google Scholar
- Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2020. Geom-gcn: Geometric graph convolutional networks. In ICLR.Google Scholar
- Erzsébet Ravasz and Albert-László Barabási. 2003. Hierarchical organization in complex networks. Physical review E 67, 2 (2003), 026112.Google Scholar
- Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In ICML. PMLR, 4334–4343.Google Scholar
- Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2019. Dropedge: Towards deep graph convolutional networks on node classification. In ICLR.Google Scholar
- Benedek Rozemberczki, Carl Allen, and Rik Sarkar. 2021. Multi-scale attributed node embedding. Journal of Complex Networks 9, 2 (2021), cnab014.Google ScholarCross Ref
- Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine 29, 3 (2008), 93–93.Google Scholar
- Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).Google Scholar
- Jayson Sia, Edmond Jonckheere, and Paul Bogdan. 2019. Ollivier-ricci curvature-based method to community detection in complex networks. Scientific reports 9, 1 (2019), 1–12.Google Scholar
- Jaeyun Song, Joonhyung Park, and Eunho Yang. 2022. TAM: Topology-Aware Margin Loss for Class-Imbalanced Node Classification. In ICML. PMLR, 20369–20383.Google Scholar
- Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Yuanxing Ning, Philip S Yu, and Lifang He. 2021. Sugar: Subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism. In Proceedings of the Web Conference 2021. 2081–2091.Google ScholarDigital Library
- Qingyun Sun, Jianxin Li, Beining Yang, Xingcheng Fu, Hao Peng, and Philip S Yu. 2023. Self-organization Preserved Graph Structure Learning with Principle of Relevant Information. In AAAI.Google Scholar
- Qingyun Sun, Jianxin Li, Haonan Yuan, Xingcheng Fu, Hao Peng, Cheng Ji, Qian Li, and Philip S Yu. 2022. Position-aware Structure Learning for Graph Topology-imbalance by Relieving Under-reaching and Over-squashing. In CIKM. Association for Computing Machinery, 1848–1857.Google Scholar
- Yanmin Sun, Andrew KC Wong, and Mohamed S Kamel. 2009. Classification of imbalanced data: A review. International journal of pattern recognition and artificial intelligence 23, 04 (2009), 687–719.Google ScholarCross Ref
- Alexandru Tifrea, Gary Bécigneul, and Octavian-Eugen Ganea. 2019. Poincare Glove: Hyperbolic Word Embeddings. In ICLR.Google Scholar
- Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, and Michael M Bronstein. 2022. Understanding over-squashing and bottlenecks on graphs via curvature. In ICLR.Google Scholar
- Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.Journal of machine learning research 9, 11 (2008).Google Scholar
- Alexei Vázquez, Romualdo Pastor-Satorras, and Alessandro Vespignani. 2002. Large-scale topological and dynamical properties of the Internet. Physical Review E 65, 6 (2002), 066130.Google ScholarCross Ref
- Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.Google Scholar
- Yu Wang, Charu Aggarwal, and Tyler Derr. 2021. Distance-wise Prototypical Graph Neural Network in Node Imbalance Classification. arXiv preprint arXiv:2110.12035 (2021).Google Scholar
- Ze Ye, Kin Sum Liu, Tengfei Ma, Jie Gao, and Chao Chen. 2019. Curvature graph network. In ICLR.Google Scholar
- Jinze Yu, Jiaming Liu, Xiaobao Wei, Haoyi Zhou, Yohei Nakata, Denis Gudovskiy, Tomoyuki Okuno, Jianxin Li, Kurt Keutzer, and Shanghang Zhang. 2022. Cross-Domain Object Detection with Mean-Teacher Transformer. In ECCV.Google Scholar
- Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2020. Deep learning on graphs: A survey. IEEE Transactions on Knowledge and Data Engineering (2020).Google ScholarDigital Library
Index Terms
- Hyperbolic Geometric Graph Representation Learning for Hierarchy-imbalance Node Classification
Index terms have been assigned to the content through auto-classification.
Recommendations
Position-aware Structure Learning for Graph Topology-imbalance by Relieving Under-reaching and Over-squashing
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge ManagementTopology-imbalance is a graph-specific imbalance problem caused by the uneven topology positions of labeled nodes, which significantly damages the performance of GNNs. What topology-imbalance means and how to measure its impact on graph learning remain ...
GraFN: Semi-Supervised Node Classification on Graph with Few Labels via Non-Parametric Distribution Assignment
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information RetrievalDespite the success of Graph Neural Networks (GNNs) on various applications, GNNs encounter significant performance degradation when the amount of supervision signals, i.e., number of labeled nodes, is limited, which is expected as GNNs are trained ...
Label-guided graph contrastive learning for semi-supervised node classification
AbstractSemi-supervised node classification is a task of predicting the labels of unlabeled nodes using limited labeled nodes and numerous unlabeled nodes. Recently, Graph Neural Networks (GNNs) have achieved remarkable success in this task. However, ...
Highlights- The framework explores semantic-level feature similarity.
- The self-checking mechanism ensures the authenticity of the positive nodes.
- The reweighting strategy enhances the effect of hard negative nodes.
- The training algorithm ...
Comments