Identification of sparse neural functional connectivity using penalized likelihood estimation and basis functions

Abstract

One key problem in computational neuroscience and neural engineering is the identification and modeling of functional connectivity in the brain using spike train data. To reduce model complexity, alleviate overfitting, and thus facilitate model interpretation, sparse representation and estimation of functional connectivity is needed. Sparsities include global sparsity, which captures the sparse connectivities between neurons, and local sparsity, which reflects the active temporal ranges of the input-output dynamical interactions. In this paper, we formulate a generalized functional additive model (GFAM) and develop the associated penalized likelihood estimation methods for such a modeling problem. A GFAM consists of a set of basis functions convolving the input signals, and a link function generating the firing probability of the output neuron from the summation of the convolutions weighted by the sought model coefficients. Model sparsities are achieved by using various penalized likelihood estimations and basis functions. Specifically, we introduce two variations of the GFAM using a global basis (e.g., Laguerre basis) and group LASSO estimation, and a local basis (e.g., B-spline basis) and group bridge estimation, respectively. We further develop an optimization method based on quadratic approximation of the likelihood function for the estimation of these models. Simulation and experimental results show that both group-LASSO-Laguerre and group-bridge-B-spline can capture faithfully the global sparsities, while the latter can replicate accurately and simultaneously both global and local sparsities. The sparse models outperform the full models estimated with the standard maximum likelihood method in out-of-sample predictions.

Appendix

Derivation of matrix R

$$ g\left(\theta (t)\right)={c}_0+\phi {(t)}^Tc $$

$$ l\left(y\Big|X\right)={\displaystyle \sum_{t=1}^Ty(t) \ln \theta (t)+\left(1-y(t)\right) \ln \left(1-\theta (t)\right)} $$

$$ \begin{array}{ll}\frac{\partial }{\partial {c}_i}l\left(y\Big|X\right)\hfill & ={\displaystyle \sum_{t=1}^T\left[\frac{y(t)}{\theta (t)}-\frac{1-y(t)}{1-\theta (t)}\right]}\left(\frac{\partial \theta (t)}{\partial {c}_i}\right)\hfill \\ {}\hfill & ={\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]}\left(\frac{\partial \theta (t)}{\partial {c}_i}\right)\hfill \end{array} $$

For logit link function, we have

$$ \begin{array}{c}\hfill \theta (t)=\frac{1}{1+ \exp \left\{-{c}_0-\phi {(t)}^Tc\right\}}\hfill \\ {}\hfill \frac{\partial \theta (t)}{\partial {c}_i}=\frac{ \exp \left\{-{c}_0-\phi {(t)}^Tc\right\}{\phi}_i(t)}{{\left(1+ \exp \left\{-{c}_0-\phi {(t)}^Tc\right\}\right)}^2}=\theta (t)\left(1-\theta (t)\right){\phi}_i(t)\hfill \\ {}\frac{\partial }{\partial {c}_i}l\left(y\Big|X\right)={\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]\left(\frac{\partial \theta (t)}{\partial {c}_i}\right)}\hfill \\ {}\hfill ={\displaystyle \sum_{t=1}^T\left[y(t)-\theta (t)\right]}{\phi}_i(t)\hfill \\ {}\frac{\partial^2}{\partial {c}_i\partial {c}_j}l\left(y\Big|X\right)=\frac{\partial }{\partial {c}_j}{\displaystyle \sum_{t=1}^T\left[y(t)-\theta (t)\right]{\phi}_i(t)}\hfill \\ {}\hfill ={\displaystyle \sum_{t=1}^T\left[-\frac{\partial }{\partial {c}_j}\theta (t)\right]}{\phi}_i(t)\hfill \\ {}\hfill =-{\displaystyle \sum_{t=1}^T\theta (t)\left(1-\theta (t)\right)}{\phi}_i(t){\phi}_j(t)\hfill \end{array} $$

where ϕ _i denotes the ith column of ϕ (denoted as v in the main text).

Thus, we have

$$ {\nabla}^2l\left(y\Big|X\right)=-{\varPhi}^T R\varPhi $$

where

$$ \varPhi ={\left(\begin{array}{ccc}\hfill {\phi}_1(1)\hfill & \hfill \cdots \hfill & \hfill {\phi}_1(T)\hfill \\ {}\hfill \vdots \hfill & \hfill \ddots \hfill & \hfill \vdots \hfill \\ {}\hfill {\phi}_{NJ}(1)\hfill & \hfill \cdots \hfill & \hfill {\phi}_{NJ}(T)\hfill \end{array}\right)}^T,R=\left(\begin{array}{ccc}\hfill \theta (1)\left(1-\theta (1)\right)\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \hfill & \hfill \theta (T)\left(1-\theta (T)\right)\hfill \end{array}\right) $$

For probit link function, we have θ(t) = F{c ₀ + ϕ(t)^T c}, where F is the normal cumulative distribution function in Eq. (5). $\frac{\partial \theta (t)}{\partial {c}_i}=f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)$, where f is the normal density function.

$$ \begin{array}{l}\begin{array}{c}\hfill \frac{\partial }{\partial {c}_i}l\left(y\Big|X\right)={\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]}\left(\frac{\partial \theta (t)}{\partial {c}_i}\right)\hfill \\ {}\hfill ={\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \\ {}\hfill \frac{\partial^2}{\partial {c}_i\partial {c}_j}l\left(y\Big|X\right)=\frac{\partial }{\partial {c}_j}{\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \\ {}\hfill ={\displaystyle \sum_{t=1}^T\frac{\partial }{\partial {c}_j}}\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \end{array}\\ {}\begin{array}{ccccc}\hfill \hfill & \hfill \hfill & \hfill \hfill & \hfill \hfill & \hfill +{\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]}\frac{\partial }{\partial {c}_j}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \end{array}\end{array} $$

Since

$$ \begin{array}{l}{\displaystyle \sum_{t=1}^T\frac{\partial }{\partial {c}_j}}\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\\ {}\begin{array}{lllll}\hfill & \hfill & \hfill & \hfill & ={\displaystyle \sum_{t=1}^T\frac{\partial }{\partial {c}_j}\left[\frac{y(t)}{\theta (t)\left(1-\theta (t)\right)}\right]}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \\ {}\hfill & \hfill & \hfill & \hfill & -{\displaystyle \sum_{t=1}^T\frac{\partial }{\partial {c}_j}\left[\frac{1}{1-\theta (t)}\right]}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \\ {}\hfill & \hfill & \hfill & \hfill & ={\displaystyle \sum_{t=1}^T\left[\frac{-y(t)\left(1-2\theta (t)\right)}{\theta {(t)}^2{\left(1-\theta (t)\right)}^2}\frac{\partial \theta (t)}{\partial {c}_j}\right]}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \\ {}\hfill & \hfill & \hfill & \hfill & -{\displaystyle \sum_{t=1}^T\left[\frac{1}{{\left(1-\theta (t)\right)}^2}\frac{\partial \theta (t)}{\partial {c}_j}\right]}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\hfill \\ {}\hfill & \hfill & \hfill & \hfill & ={\displaystyle \sum_{t=1}^T\left[\frac{-y(t)\left(1-2\theta (t)\right)}{\theta {(t)}^2{\left(1-\theta (t)\right)}^2}\right]}f{\left\{{c}_0+\phi {(t)}^Tc\right\}}^2{\phi}_i(t){\phi}_j(t)\hfill \\ {}\hfill & \hfill & \hfill & \hfill & -{\displaystyle \sum_{t=1}^T\left[\frac{1}{{\left(1-\theta (t)\right)}^2}\right]}f{\left\{{c}_0+\phi {(t)}^Tc\right\}}^2{\phi}_i(t){\phi}_j(t)\hfill \\ {}\hfill & \hfill & \hfill & \hfill & ={\varPhi}^T{R}_1\varPhi +{\varPhi}^T{R}_2\varPhi \hfill \end{array}\end{array} $$

where

$$ \begin{array}{c}\hfill {R}_1=\left(\begin{array}{ccc}\hfill \left[\frac{-y(1)\left(1-2\theta (1)\right)}{\theta {(1)}^2{\left(1-\theta (1)\right)}^2}\right]f{\left\{{c}_0+\phi {(1)}^Tc\right\}}^2\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \hfill & \hfill \left[\frac{-y(T)\left(1-2\theta (T)\right)}{\theta {(T)}^2{\left(1-\theta (T)\right)}^2}\right]f{\left\{{c}_0+\phi {(T)}^Tc\right\}}^2\hfill \end{array}\right)\hfill \\ {}\hfill {R}_2=\left(\begin{array}{ccc}\hfill -\left[\frac{1}{{\left(1-\theta (1)\right)}^2}\right]f{\left\{{c}_0+\phi {(1)}^Tc\right\}}^2\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \hfill & \hfill -\left[\frac{1}{{\left(1-\theta (T)\right)}^2}\right]f{\left\{{c}_0+\phi {(T)}^Tc\right\}}^2\hfill \end{array}\right)\hfill \end{array} $$

In addition, we have

$$ \begin{array}{l}{\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]}\frac{\partial }{\partial {c}_j}f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t)\\ {}\begin{array}{ccc}\hfill \begin{array}{cc}\hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill & \hfill \hfill \end{array}\hfill & \hfill \hfill & \hfill \hfill \end{array}=-{\displaystyle \sum_{t=1}^T\left[\frac{y(t)-\theta (t)}{\theta (t)\left(1-\theta (t)\right)}\right]}\left({c}_0+\phi {(t)}^Tc\right)f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t){\phi}_j(t)\\ {}\begin{array}{ccc}\hfill \begin{array}{cc}\hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill & \hfill \hfill \end{array}\hfill & \hfill \hfill & \hfill \hfill \end{array}=-{\displaystyle \sum_{t=1}^T\left[\frac{y(t)}{\theta (t)\left(1-\theta (t)\right)}\right]}\left({c}_0+\phi {(t)}^Tc\right)f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t){\phi}_j(t)\\ {}\begin{array}{ccc}\hfill \begin{array}{cc}\hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill & \hfill \hfill \end{array}\hfill & \hfill \hfill & \hfill \hfill \end{array}+{\displaystyle \sum_{t=1}^T\left[\frac{1}{1-\theta (t)}\right]}\left({c}_0+\phi {(t)}^Tc\right)f\left\{{c}_0+\phi {(t)}^Tc\right\}{\phi}_i(t){\phi}_j(t)\\ {}\begin{array}{cccc}\hfill \begin{array}{cc}\hfill \hfill & \hfill \hfill \end{array}\hfill & \hfill \hfill & \hfill \hfill & \hfill \hfill \end{array}={\varPhi}^T{R}_3\varPhi +{\varPhi}^T{R}_4\varPhi \end{array} $$

Where

$$ \begin{array}{c}\hfill {R}_3=\left(\begin{array}{ccc}\hfill -\left[\frac{y(1)}{\theta (1)\left(1-\theta (1)\right)}\right]\left({c}_0+\phi {(1)}^Tc\right)f\left\{{c}_0+\phi {(1)}^Tc\right\}\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \hfill & \hfill -\left[\frac{y(T)}{\theta (T)\left(1-\theta (T)\right)}\right]\left({c}_0+\phi {(T)}^Tc\right)f\left\{{c}_0+\phi {(T)}^Tc\right\}\hfill \end{array}\right)\hfill \\ {}\hfill {R}_4=\left(\begin{array}{ccc}\hfill \left[\frac{1}{1-\theta (1)}\right]\left({c}_0+\phi {(1)}^Tc\right)f\left\{{c}_0+\phi {(1)}^Tc\right\}\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill \hfill & \hfill \hfill & \hfill \left[\frac{1}{1-\theta (T)}\right]\left({c}_0+\phi {(T)}^Tc\right)f\left\{{c}_0+\phi {(T)}^Tc\right\}\hfill \end{array}\right)\hfill \end{array} $$

Thus, we have

∇ ² l(y|X) = Φ ^T RΦ, where R = R ₁ + R ₂ + R ₃ + R ₄.