Recovering Communities in Temporal Networks Using Persistent Edges

Abstract

This article studies the recovery of static communities in a temporal network. We introduce a temporal stochastic block model where dynamic interaction patterns between node pairs follow a Markov chain. We render this model versatile by adding degree correction parameters, describing the tendency of each node to start new interactions. We show that in some cases the likelihood of this model is approximated by the regularized modularity of a time-aggregated graph. This time-aggregated graph involves a trade-off between new edges and persistent edges. A continuous relaxation reduces the regularized modularity maximization to a normalized spectral clustering. We illustrate by numerical experiments the importance of edge persistence, both on simulated and real data sets.

This work has been done within the project of Inria - Nokia Bell Labs “Distributed Learning and Control for Network Analysis” and was partially supported by COSTNET Cost Action CA15109.

A Proofs of Main Statements

1.1 A.1 Maximum Likelihood Computations (Proposition 1)

Proof

(Proof of Proposition 1). By the temporal Markov property, the log-likelihood of the model can be written as $\log \mathbb {P}(A \, | \,Z,\theta ) = \log \mathbb {P}(A^1 \, | \,Z,\theta ) + \sum _{t=2}^T \mathbb {P}(A^t \, | \,A^{t-1},Z,\theta )$. By denoting $\rho _a^{\theta _i\theta _j} = \log \frac{\mu _a^{\theta _i\theta _j}}{\nu _a^{\theta _i\theta _j}}$, we find that

$$\begin{aligned} \log \mathbb {P}(A^1 \, | \,Z, \theta )&\ = \ \frac{1}{2} \sum _{i,j} \sum _a \delta (A_{ij}^1, a) \Big ( \delta (Z_i,Z_j) \rho _a^{\theta _i\theta _j} + \log \nu _a^{\theta _i\theta _j} \Big ) \\&\ = \ \frac{1}{2} \sum _{i,j} \delta (Z_i,Z_j) \sum _a \delta (A_{ij}^1, a) \rho _a^{\theta _i\theta _j} + c_1(A), \end{aligned}$$

where $c_1(A) = \frac{1}{2} \sum _{i,j} \sum _a \delta (A_{ij}^1, a) \log \nu _a^{\theta _i\theta _j}$ does not depend on the community structure. Similarly, by denoting $R^{\theta _i\theta _j}_{ab} = \log \frac{P_{ab}^{\theta _i\theta _j}}{Q_{ab}^{\theta _i\theta _j}}$ we find that

$$\begin{aligned} \log \mathbb {P}(A^t \, | \,A^{t-1}, Z, \theta )&\ = \ \frac{1}{2} \sum _{i,j} \sum _{a,b} \delta (A_{ij}^{t-1}, a) \delta (A_{ij}^t, b) \Big ( \delta (Z_i,Z_j) R_{ab}^{\theta _i\theta _j} + \log Q_{ab}^{\theta _i\theta _j} \Big ) \\&\ = \ \frac{1}{2} \sum _{i,j} \delta (Z_i,Z_j) \sum _{a,b} \delta (A_{ij}^{t-1}, a) \delta (A_{ij}^t, b) R_{ab}^{\theta _i\theta _j} + c_t(A), \end{aligned}$$

where $c_t(A) = \frac{1}{2} \sum _{i,j} \sum _{a,b} \delta (A_{ij}^{t-1}, a) \delta (A_{ij}^t, b) \log Q_{ab}^{\theta _i\theta _j}$ does not depend on the community structure. Simple computations show that

$$ \sum _a \delta (A_{ij}^1, a) \rho _a^{\theta _i\theta _j} \ = \ A_{ij}^1 (\rho _1^{\theta _i\theta _j}-\rho _0^{\theta _i\theta _j}) + \rho _0^{\theta _i\theta _j} $$

and

$$\begin{aligned} \sum _{a,b} \delta (A_{ij}^{t-1}, a) \delta (A_{ij}^t, b) R_{ab}^{\theta _i\theta _j}&\ = \ R_{00}^{\theta _i\theta _j} + A_{ij}^{t-1} \big (R_{10}^{\theta _i\theta _j} - R_{00}^{\theta _i\theta _j} \big ) + A_{ij}^t \big (R_{01}^{\theta _i\theta _j} - R_{00}^{\theta _i\theta _j} \big ) \\&\qquad + A_{ij}^{t-1} A_{ij}^t \big ( R_{11}^{\theta _i\theta _j} - R_{01}^{\theta _i\theta _j} - R_{10}^{\theta _i\theta _j} + R_{00}^{\theta _i\theta _j} \big ) \\&\ = \ R_{00}^{\theta _i\theta _j} + A_{ij}^{t-1} \ell _{10}^{\theta _i\theta _j} + A_{ij}^{t} \ell _{01}^{\theta _i\theta _j} + A_{ij}^{t-1} A_{ij}^t \big ( \ell _{11}^{\theta _i\theta _j} - \ell _{01}^{\theta _i\theta _j} - \ell _{10}^{\theta _i\theta _j} \big ). \end{aligned}$$

By collecting the above observations, we now find that $\log \mathbb {P}(A \, | \,Z, \theta )$ equals

$$\begin{aligned} c(A) +&\frac{1}{2}\sum _{i,j} \delta (Z_i,Z_j) \Bigg \{ A_{ij}^{1} (\rho _1^{\theta _i\theta _j} - \rho _0^{\theta _i\theta _j}) + \rho _0^{\theta _i\theta _j} + (A_{ij}^{1} - A_{ij}^{T}) \ell _{10}^{\theta _i\theta _j} \Bigg \} \\ +&\frac{1}{2} \sum _{i,j} \delta (Z_i,Z_j) \sum _{t=2}^{T} \Bigg \{ ( \ell _{01}^{\theta _i\theta _j} + \ell _{10}^{\theta _i\theta _j} ) \left( A_{ij}^{t} - A_{ij}^{t-1} A_{ij}^{t} \right) + \ell _{11}^{\theta _i\theta _j} A_{ij}^{t-1} A_{ij}^{t} - \log \frac{Q_{00}^{\theta _i\theta _j}}{P_{00}^{\theta _i\theta _j}} \Bigg \}, \end{aligned}$$

where $c(A) = \sum _t c_t(A)$ does not depend on Z. Hence the claim follows.    $\square$

1.2 A.2 Approximation of the MLE

Recall the structural assumptions (3)–(4) about the degree correction parameters. Because $P_{01}, Q_{01} = o(1)$, a first-order Taylor expansion yields

$$\begin{aligned} \log \frac{ 1 - \theta _i \theta _j Q_{01} }{ 1 - \theta _i \theta _j P_{01} } \ = \ \theta _i \theta _j \left( P_{01} - Q_{01} \right) + o\left( P_{01}^2 + Q_{01}^2 \right) \ = \ \frac{ \bar{d} _i \bar{d} _j }{2 \bar{m} } + o\left( P_{01}^2 + Q_{01}^2 \right) , \end{aligned}$$

as well as $\ell _{01}^{\theta _i\theta _j} \approx \log \frac{P_{01}}{Q_{01}}$, $\ell _{10}^{\theta _i\theta _j} \approx \log \frac{ 1 - P_{11} }{ 1 - Q_{11} }$ and $\ell _{11}^{\theta _i\theta _j} \approx \log \frac{P_{11}}{Q_{11}}$. Using these approximations in the MLE expression leads to the maximisation of

$$\begin{aligned} \sum _{t=2}^{T} \sum _{ i,j :z_i = z_j } \left( \tilde{a}_{ij}^{t} - \theta _i \theta _j \left( P_{01} - Q_{01} \right) \right) . \end{aligned}$$

where $\tilde{a}_{ij}^{t} = \alpha \left( A^t_{ \mathrm {new} } \right) _{ij} + \beta \left( A_{ \mathrm {pers} }^t \right) _{ij}$. Since $\mu$ and $\nu$ are stationary distributions,

$$\begin{aligned} \mathbb {E}\left( A_{ \mathrm {new} }^{t} \right) _{ij}&\ = \ {\left\{ \begin{array}{ll} \theta _i \theta _j \mu _1 (1-P_{11}) &{} \text { if } Z_i = Z_j \\ \theta _i \theta _j \nu _1(1-Q_{11} ) &{} \text { otherwise,} \end{array}\right. } \\ \mathbb {E}\left( A_{ \mathrm {pers} }^{t} \right) _{ij}&\ = \ {\left\{ \begin{array}{ll} \theta _i \theta _j \mu _1 P_{11} &{} \text { if } Z_i = Z_j \\ \theta _i \theta _j \nu _1Q_{11} &{} \text { otherwise.} \end{array}\right. } \end{aligned}$$

Therefore, using $W_{ij} = \sum _{t=2}^T \tilde{a}_{ij}$ we have

$$\begin{aligned} \mathbb {E}W_{ij} \ = \ {\left\{ \begin{array}{ll} (T-1) \theta _i \theta _j \mu _1 \left( \alpha (1-P_{11}) + \beta P_{11} \right) &{} \text { if } Z_i = Z_j \\ (T-1) \theta _i \theta _j \nu _1 \left( \alpha (1-Q_{11}) + \beta Q_{11} \right) &{} \text { otherwise.} \end{array}\right. } \end{aligned}$$

Since the community labeling are sampled uniformly at random and using the normalization for the $\theta _i$’s, we have

$$\begin{aligned} \bar{d} _i \ = \ (T-1) \theta _i N \, \frac{ \mu _1 \left( \alpha (1-P_{11}) + \beta P_{11} \right) + (K-1) \nu _1 \left( \alpha (1-Q_{11}) + \beta Q_{11} \right) }{ K } \\ \end{aligned}$$

together with $\bar{m} = \frac{N^2}{2} \frac{ \mu _1 \left( \alpha (1-P_{11}) + \beta P_{11} \right) + (K-1) \nu _1 \left( \alpha (1-Q_{11}) + \beta Q_{11} \right) }{ K }.$    $\square$

1.3 A.3 Modularity and Normalized Spectral Clustering

The regularized modularity of a partition $Z \in [K]^N$ of the graph A is defined as

$$\begin{aligned} \mathcal {M}\left( A, Z, \gamma \right) \ = \ \sum _{i,j} \delta \left( Z_{i}, Z_{j} \right) \left( A_{ij} - \gamma \frac{d_i d_j}{2m} \right) \end{aligned}$$

where $d = A 1_n$ and $\gamma$ is a resolution parameter. This can be rewritten as

$$\begin{aligned} \mathcal {M}\left( A, Z, \gamma \right) \ = \ \mathrm {Tr}\; \tilde{Z} ^T \left( A - \gamma \frac{d d^T}{2m} \right) \tilde{Z} \end{aligned}$$

where $\tilde{Z} \in \{0,1\}^{N\times K}$ is the membership matrix associated to the vector Z, that is $\tilde{Z} _{ik} = 1$ for $k = Z_i$, and $\tilde{Z} _{ik} = 0$ otherwise. As maximizing the modularity over $Z \in \mathcal {Z}_{N,K}$ is in general NP-complete [4], it is convenient to perform a continuous relaxation. Following [17], we transform the problem into

$$\begin{aligned} \hat{X} \ = \ \mathop {\mathrm {arg\,max}}\limits _{ \begin{array}{c} X \in \mathbb {R}^{N\times K} \\ X^T D X = I_K \end{array} } \mathrm {Tr}\;X^T \left( A - \gamma \frac{d d^T}{2m} \right) X. \end{aligned}$$

(8)

The predicted membership matrix $\hat{Z}$ is then recovered by performing an approximated solution to the following k-means problem (see [12])

$$\begin{aligned} \left( \hat{Z} , \hat{Y} \right) \ = \ \mathop {\mathrm {arg\,min}}\limits _{ Z \in \mathcal {Z}_{N,K}, Y \in \mathbb {R}^{K\times K} } \left\| Z Y - \hat{X} \right\| _F. \end{aligned}$$

(9)

The Lagrangian associated to the optimization problem (8) is

$$\begin{aligned} \mathrm {Tr}\;X^T \left( A - \gamma \frac{d d^T}{2m} \right) X - \mathrm {Tr}\;\left( \varLambda ^T \left( X^T D X - I_K\right) \right) \end{aligned}$$

where $\varLambda \in \mathbb {R}^{K\times K}$ is a symmetric matrix of Lagrangian multipliers. Up to a change of basis, we can assume that $\varLambda$ is diagonal. The solution of (8) verifies

$$\begin{aligned} \left( A - \gamma \frac{d d^T}{2m} \right) X \ = \ D X \varLambda \quad \text { and } \quad X^T D X \ = \ I_K, \end{aligned}$$

which is a generalized eigenvalue problem: the columns of X are the generalized eigenvectors, and the diagonal elements of $\varLambda$ are the eigenvalues. In particular, since the constant vector $1_n$ verifies $(A - \gamma \frac{d d^T}{2m} ) 1_n = (1-\gamma ) D 1_n$, we conclude that the eigenvalues should be larger than $1-\gamma$ for the partition to be meaningful.

Multiplying the first equation by $1_n^T$ leads to $(1-\gamma ) d^T X = d^T X \varLambda$, and therefore $d^T X = 0$ (using the previous remark on $\varLambda$). The system then simplifies in

$$\begin{aligned} A X \ = \ D X \varLambda \quad \text { and } \quad X^T D X \ = \ I_K. \end{aligned}$$

Defining a re-scaled vector $U = D^{-1/2}X$ shows that U verifies $D^{-1/2}A D^{-1/2} U = U \varLambda$ and $U^T U = I_K$. Thus, the columns of U are eigenvectors of $D^{-1/2}A D^{-1/2}$ associated to the K largest eigenvalue (or equivalently, the eigenvectors of $\mathcal {L}= I_N - D^{-1/2}A D^{-1/2}$ associated to the K smallest eigenvalues).