HyperDNE: Enhanced hypergraph neural network for dynamic network embedding

Abstract

Representation learning provides an attractive opportunity to model the evolution of dynamic networks. However, the existing methods have two limitations: (1) most graph neural network-based methods fail to utilize the high-order proximity of nodes that captures the important properties of a network topology; (2) evolutionary dynamics-based methods are much fine-grained in modeling time information but neglect the coherence of dynamic networks, which leads to the model being susceptible to subtle noise. In this paper, we propose an enhanced hypergraph neural network framework for dynamic network embedding (HyperDNE) to tackle these issues. Specifically, we innovatively design a sequential hypergraph with dual-stream output to explore the group properties of nodes and edges, and a line graph neural network is added as an auxiliary enhancement scheme to further aggregate social influence from the degree of social convergence. Then, we compute the final embedding through attentions along the node and hyperedge levels to fuse multi-level variations in the network structure. The experimental results on six real networks demonstrate significant gains for HyperDNE over several state-of-the-art network embedding baselines. The dataset and source code of HyperDNE are publicly available at https://github.com/qhgz2013/HyperDNE.

Introduction

Network embedding, also known as network representation learning, aims to learn low-dimensional representations of nodes to capture the structural information of a network. Recent work on network embedding has been widely used in many fields, such as recommendation systems [38], natural language processing [7] and computational biology [25].

In general, most existing network embedding methods have mainly been designed for static graphs with fixed nodes and edges that occur during the same period. For example, Node2vec [13] and GraphSAGE [14] explore topology or use node attribute information to learn static representations. However, networks in the real world exhibit complex temporal properties, which means that the network structures grow over time rather than appear overnight. Such dynamic networks are usually represented as graphical snapshots at different time intervals [23]. As a natural attribute of a real network, natural temporal dimension information cannot be ignored; otherwise, network embedding cannot preserve dynamic network structures well.

Learning information-rich network representations on dynamic networks is extremely challenging, and it is not until recently that several solutions have been proposed. Such embedding, which encodes the dynamic graph structure, can benefit temporal link prediction. For example, given a social network, we would like to predict power shifts in political networks or identify potential criminal organizations. Following the success of graph neural networks(GNNs), many nascent efforts have shown a significant performance improvement by concentrating on minimizing the loss function of a certain prediction task to represent the whole graph [9], [49]. However, the real-world networks are affected by short-term irrelevant nodes, resulting in GNNs learning suboptimal representation. Graph attention networks [31], [30], [41] alleviate this issue by capturing how important each neighbor in the graph represents the central node. However, in social networks, people of one mind fall into the same group, and such cluster relationships are usually associated with higher-order relationships as opposed to pairwise relationships. These methods show performance improvements, but their use of simple graph structure determines them to deal with only the node interactions of a pair of coupling relations during the evolution of the network, which limits the scope of the structural properties to low-order approximations. Several evolutionary dynamics-based approaches(e.g., the Hawkes process [50] and the triadic closure process [47]) model the time process through the interaction mechanism between nodes to simulate the evolution and nature of the network. However, these methods are so fine-grained in dealing with time information, and such dynamic models will make short-term user correlations susceptible to these noise correlations when users encounter friends by chance at a certain time. In recent years, p-LapR [22] has been proposed as a natural generalization of standard graph Laplacian to achieve scene recognition to better preserve the local structure. PLapHGNN [24] propose P-Laplacian-based hypergraph neural networks to classify nonstructural data using optimized higher-order manifolds of data. HpLapGCN [10] exploits valid GCN variants to accurately express the structural information of the data. These methods will not change the structure information of samples once they are constructed according to the original data, i.e., static high-order data represents the learning structure.

Overall, in learning dynamic networks, the following challenges remain:

High-order node proximity preservation (Challenge 1): The evolution of a dynamic network is the dynamic change process of node interactions. Therefore, dynamic networks show the group properties of nodes and edges; i.e., the generation of links is triggered by the joint effect of the previous node neighborhood structure and social circle. This cluster-aware high-order proximity needs to be well examined and preserved.

Network coherence preservation (Challenge 2): Styles of social interactions tend to remain relatively stable in the short term, so the evolution of dynamic social networks is not strictly time-dependent; instead, it tends to be sequentially dependent. Existing methods are so fine-grained in dealing with the time information of nodes but ignore the coherence of the whole network, which may lead to a large deviation in the prediction of interactions in the next stage.

To address the above challenges, we propose an enhanced hypergraph neural network framework for dynamic network embedding (HyperDNE). We innovatively design a sequential hypergraph with dual-stream output to explore the group properties of nodes and edges. In addition, to cooperate with hypergraphs and further aggregate social influences, we introduce the line graph neural network [40], [3]. The edges of the line graph show the high-order proximity of the cluster perception. (Challenge #1). To fuse multi-level variations in network structure, we learn multiple attention at the node level and hyperedge level to achieve joint attention on different latent subspaces to preserve the coherence of the dynamic network. (Challenge #2).

The main contributions of this work are summarized as follows:

• We investigate the joint potential of hypergraph modeling and line graph neural networks in preserving the high-order proximity of cluster awareness by exploring the group properties represented by nodes and edges in dynamic networks. Our work is among the earliest works that study both hypergraphs and line graphs on dynamic graphs.

• To fuse multi-level variations in network structure, we learn multiple attentions at the node level and hyperedge level to achieve joint attention on different latent subspaces to preserve the coherence of the dynamic network.

• We conduct extensive experiments to demonstrate that the model consistently outperforms the most advanced methods for both single-step and multi-step link prediction tasks and ablate the model to analyze the validity of our model components.