Qingqing Long
ABSTRACT
To depict ubiquitous relational data in real world, network data have been widely applied in modeling complex relationships. Projecting vertices to low dimensional spaces, quoted as Network Embedding, would thus be applicable to diverse real-world predicative tasks. Numerous works exploiting pairwise proximities, one characteristic owned by real networks, the clustering property, namely vertices are inclined to form communities of various ranges and hence form a hierarchy consisting of communities, has barely received attention from researchers. In this paper, we propose our network embedding framework, abbreviated SpaceNE, preserving hierarchies formed by communities through subspaces, manifolds with flexible dimensionalities and are inherently hierarchical. Moreover, we propose that subspaces are able to address further problems in representing hierarchical communities, including sparsity and space warps. Last but not least, we proposed constraints on dimensions of subspaces to denoise, which are further approximated by differentiable functions such that joint optimization is enabled, along with a layer-wise scheme to alleviate the overhead cause by the vast number of parameters. We conduct various experiments with results demonstrating our model's effectiveness in addressing community hierarchies.
- Åke Björck. Numerics of gram-schmidt orthogonalization. Linear Algebra and Its Applications, 197:297--316, 1994.Google Scholar
Cross Ref
- Emmanuel J Candes and Benjamin Recht. Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6):717--772, 2009.Google ScholarDigital Library
- Aaron Clauset, Cristopher Moore, and M E J Newman. Structural inference of hierarchies in networks. international conference on machine learning, pages 1--13, 2006.Google ScholarDigital Library
- Aaron Clauset, Cristopher Moore, and Mark EJ Newman. Hierarchical structure and the prediction of missing links in networks. Nature, 453(7191):98, 2008.Google ScholarCross Ref
- Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering, pages 1--1, 2018.Google Scholar
- Chris Ding, Ding Zhou, Xiaofeng He, and Hongyuan Zha. R 1-pca: rotational invariant l 1-norm principal component analysis for robust subspace factorization. In Proceedings of the 23rd international conference on Machine learning, pages 281--288. ACM, 2006.Google ScholarDigital Library
- Lun Du, Zhicong Lu, Yun Wang, Guojie Song, Yiming Wang, and Wei Chen. Galaxy network embedding: a hierarchical community structure preserving approach. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 2079--2085. AAAI Press, 2018.Google ScholarCross Ref
- Lun Du, Yun Wang, Guojie Song, Zhicong Lu, and Junshan Wang. Dynamic network embedding: an extended approach for skip-gram based network embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 2086--2092. AAAI Press, 2018.Google ScholarCross Ref
- Francois Fouss, Alain Pirotte, Jeanmichel Renders, and Marco Saerens. Randomwalk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Transactions on Knowledge and Data Engineering, 19(3):355--369, 2007.Google ScholarDigital Library
- Michelle Girvan and Mark EJ Newman. Community structure in social and biological networks. Proceedings of the national academy of sciences, 99(12):7821-- 7826, 2002.Google ScholarCross Ref
- Robert Hechtnielsen. Theory of the backpropagation neural network. Neural Networks, 1:445--448, 1988.Google ScholarCross Ref
- Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504--507, 2006.Google Scholar
- Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In Advances in neural information processing systems, pages 2177-- 2185, 2014.Google ScholarDigital Library
- Ziyao Li, Liang Zhang, and Guojie Song. Sepne: Bringing separability to network embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4261--4268, 2019.Google ScholarCross Ref
- Guangcan Liu, Zhouchen Lin, and Yong Yu. Robust subspace segmentation by low-rank representation. In Proceedings of the 27th international conference on machine learning (ICML-10), pages 663--670, 2010.Google ScholarDigital Library
- Laurens Van Der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(2605):2579--2605, 2008.Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.Google Scholar
- Yurii Nesterov. Smooth minimization of non-smooth functions. Mathematical Programming, 103(1):127--152, 2005.Google ScholarDigital Library
- M E J Newman and Michelle Girvan. Finding and evaluating community structure in networks. Physical Review E, 69(2):026113--026113, 2004.Google ScholarCross Ref
- Mark EJ Newman. The structure and function of complex networks. SIAM review, 45(2):167--256, 2003.Google ScholarDigital Library
- M E J Newman. Finding community structure in networks using the eigenvectors of matrices. Physical Review E, 74(3):036104, 2006.Google ScholarCross Ref
- Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations. In Advances in neural information processing systems, pages 6338--6347, 2017.Google Scholar
- Bryan Perozzi, Rami Alrfou, and Steven Skiena. Deepwalk: online learning of social representations. Knowledge Discovery and Data mining, pages 701--710, 2014.Google ScholarDigital Library
- Leonardo Filipe Rodrigues Ribeiro, Pedro H P Saverese, and Daniel R Figueiredo. struc2vec : Learning node representations from structural identity. knowledge discovery and data mining, pages 385--394, 2017.Google Scholar
- Huawei Shen, Xueqi Cheng, Kai Cai, and Mao Bin Hu. Detect overlapping and hierarchical community structure in networks. Physica A Statistical Mechanics & Its Applications, 388(8):1706--1712, 2009.Google ScholarCross Ref
- Victor Spirin and Leonid A Mirny. Protein complexes and functional modules in molecular networks. Proceedings of the National Academy of Sciences of the United States of America, 100(21):12123--12128, 2003.Google ScholarCross Ref
- Gilbert Strang, Gilbert Strang, Gilbert Strang, and Gilbert Strang. Introduction to linear algebra, volume 3. Wellesley-Cambridge Press Wellesley, MA, 1993.Google Scholar
- Lei Tang and Huan Liu. Leveraging social media networks for classification. Data Mining and Knowledge Discovery, 23(3):447--478, 2011.Google ScholarDigital Library
- Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. Line: Large-scale information network embedding. In International Conference on World Wide Web, pages 1067--1077, 2015.Google ScholarDigital Library
- Amanda L. Traud, Peter J. Mucha, and Mason A. Porter. Social structure of facebook networks. Social Science Electronic Publishing, 391(16):4165--4180, 2012.Google Scholar
- René Vidal. Subspace clustering. IEEE Signal Processing Magazine, 28(2):52--68, 2011.Google ScholarCross Ref
- Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierreantoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 11:3371--3408, 2010.Google ScholarDigital Library
- Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, and Shiqiang Yang. Community preserving network embedding. In Association for the Advancement of Artificial Intelligence Conference, 2017.Google ScholarCross Ref
- JunshanWang, Zhicong Lu, Guojia Song, Yue Fan, Lun Du, andWei Lin. Tag2vec: Learning tag representations in tag networks. In The World Wide Web Conference, pages 3314--3320. ACM, 2019.Google Scholar
- Svante Wold, Kim Esbensen, and Paul Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1--3):37--52, 1987.Google Scholar
- Zi Yin and Yuanyuan Shen. On the dimensionality of word embedding. neural information processing systems, pages 895--906, 2018.Google Scholar
- Yizhou Zhang, Guojie Song, Lun Du, Shuwen Yang, and Yilun Jin. Dane: Domain adaptive network embedding. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 2019.Google ScholarCross Ref
Index Terms
- Hierarchical Community Structure Preserving Network Embedding: A Subspace Approach
Recommendations
Network Embedding on Hierarchical Community Structure Network
Network embedding is a method of learning a low-dimensional vector representation of network vertices under the condition of preserving different types of network properties. Previous studies mainly focus on preserving structural information of vertices ...
Attributed network embedding via subspace discovery
AbstractNetwork embedding aims to learn a latent, low-dimensional vector representations of network nodes, effective in supporting various network analytic tasks. While prior arts on network embedding focus primarily on preserving network topology ...
Community-aware dynamic network embedding by using deep autoencoder
Highlights- This paper studies the embedding problem (DNE) in dynamic networks.
- We design a ...
AbstractNetwork embedding has recently attracted lots of attention due to its wide applications on graph tasks such as link prediction, network reconstruction, node stabilization, and community stabilization, which aims to learn the low-...
Comments