Pattern separation

The idea of pattern separation as a desirable computational function arises from the work of David Marr on the encoding of memories [15, 16]. Marr showed that a strongly recurrent neuronal network could learn to reproduce complete patterns from fragmented inputs, but that such fragments should be distinctive to avoid catastrophic interference between different patterns. Although the theory was developed for the cerebellum, another site of pattern completion was proposed to be the CA3 hippocampal subfield, a region that displays the necessary recurrent connectivity and receives strong inputs directly from the dentate gyrus [17, 18]. Further theoretical work, in particular that of McNaughton and Morris [19] and Rolls [20], showed that pattern completion would ideally be preceded by pattern separation, originally envisioned as an orthogonalisation of input spiking vectors, and that this could take place in the dentate gyrus. The theory of pattern separation, and its occurrence in the dentate gyrus, has since been extensively developed in a large number of theoretical papers [21–26] and observed in many experiments [27–31] (for reviews see [32], [14], and [33]). Pattern separation has been viewed both in terms of the spiking outputs of the dentate gyrus and the ability of an experimental subject to discriminate between similar situations [34, 35]. Of particular clinical relevance, the dentate gyrus undergoes relatively well-described physiological changes under pathologies such as Alzheimer’s disease [36], schizophrenia [37], and epilepsy [38–40], and these conditions have direct behavioural correlates in terms of the reduced ability of sufferers to distinguish between similar situations [41–43].

A number of measures of spike train similarity have been used to assess pattern separation in the literature. The original definitions of patterns separation considered orthogonalisation as most beneficial in terms of reducing interference during pattern completion [19, 20]; this is computed as the cosine distance between (normalised) discretised spike-train vectors. Decorrelation has also been used as a natural measure of similarity reduction [44, 45] and again is computed on discretised spike-train vectors. Finally, different measures of distances between spike trains can be used to assess pattern separation; the measures can be either discretised, such as the Hamming distance [46–49], or calculated on raw spike times [50–53].

Despite being conceptually straightforward, the quantification of pattern separation in terms of a network’s inputs and outputs can be surprisingly problematic. When considered as part of a system preceding pattern completion, good pattern separation can be simply taken as any procedure that reduces noise and enhances the storage capacity of the downstream network [54]. When isolated from pattern completion, however, measurement of pattern separation efficacy can become more arbitrary. Different research questions, theoretical models, and neural codes can lead to wildly different conclusions about pattern separation. This has become particularly relevant as more detailed computational models have focussed on the dentate gyrus in order to assess the contribution of different circuit components to pattern separation [45, 46, 48, 49], and as new experimental techniques have allowed direct access to the inputs and outputs of individual dentate granule cells [44, 55]. A contributing factor to the number of different metrics is the gap between complex in vivo physiological studies which focus on the mechanisms generating pattern separation [56, 57] and more computational studies that focus on the consequences of pattern separation [58]. The former tend to dominate the literature. Different quantifications of the effect called pattern separation can lead to seemingly inconsistent conclusions, a point already noted by Santoro [34], Vineyard et al [59], and Chavlis and Poirazi [33]. Madar et al [52] showed explicitly that different measures can produce different results on the same datasets. Myers and Scharfman [46] found that removing hilar cells led to less sparse granule cell activity and improved pattern separation performance, as measured with the raw Hamming distance, a counterintuitive result given the theoretically positive effects of sparsity on pattern separation [60, 61]. Later studies that used the Hamming distance sought to correct for the relative sparsity of the outputs compared to the inputs in various ways [47, 49]. Sparsity in general can be a problem for similarity metrics as firing rates can bias correlations and cosine distances [51, 52, 62]. A particular issue arises in the choice of bin sizes to move from spike times to the discretised spike-vectors necessary to compute orthogonalisations, decorrelations, or Hamming distances; Madar et al [52] also showed that choosing different bin sizes could qualitatively change the apparent behaviour of a system, with smaller time bins typically showing pattern separation and larger time bins showing pattern convergence on the same input and output sets of spike times.

Finally, making structured inputs less similar can be achieved by destroying structure and this may degrade the ability of downstream networks, such as the CA3 subfield, to reconstruct salient features from the inputs. Vineyard et al [59] and Madar et al [44] both note that standard pattern separation measures do not consider destruction of information. This has lead to debates about the roles of different features of the dentate gyrus in pattern separation. Aimone et al [63] and Sahay et al [64] took explicitly opposing views on the function of neurogenesis in the dentate gyrus, with the former arguing that ‘memory resolution’, the transfer of information salient to memories, was the key role of more broadly-tuned adult-born neurons and the latter arguing that more excitable adult-born neurons primarily drove recurrent inhibition that increased sparsity and hence classical pattern separation. The two objectives, reliable transmission and separation through sparsity, were considered to be largely antithetical. Inconsistencies in the conception and measurement of pattern separation contribute to the continuation of this debate; later arguments have been made that adult-born granule cells aid pattern separation [59, 65, 66], hinder pattern separation [67], differentially separate the most similar patterns [68], or independently carry out the entirety of pattern separation, with mature cells responsible for behavioural pattern completion in behavioural tasks [69] (but see [70] for criticism of this idea). Some studies consider the loss of signal explicitly. Guzman et al [45], for example, trained an artificial neural network to identify input patterns from modelled dentate gyrus outputs, whilst quantifying pattern separation itself by decorrelation until the stage that inputs cannot be reliably identified by the artificial network. This is an effective solution to the problem of complete loss of signal, but leaves open questions about how close the network is to failing, or, conversely, if a more complex artificial network could recover more input patterns for a given level of decorrelation. We believe that a direct measurement of the information contained in spike train ensembles going into and out of the dentate gyrus will help to resolve these inconsistencies and be a valuable and robust assessor of dentate gyrus functionality.

Information theory in neuroscience

Information theory is built upon the work of Shannon [71] and studies the ability of systems to communicate reliably. There is a long history of the application of information theoretic techniques to neuroscience, starting with MacKayand McCulloch [72], and a large number of techniques have been developed to assess or estimate the informational content of neural codes [73–76] (see [77] or [78], among many others, for reviews). Since information has a natural link to the functions of the hippocampus in terms of memory, navigation, and situational discrimination, a number of studies have used techniques from information theory to analyse hippocampal structure and function. Treves and Rolls [79] showed that the CA3 network should receive two streams of input (from the dentate gyrus and directly from the upstream entorhinal cortex) in order to both learn new patterns and reliably access stored memories. Cerasti and Treves [80] found that the spatial representations induced in CA3 by dentate gyrus activation potentially contain much information that is so high-dimensional that it cannot be feasibly decoded. Petrantonakis and Poirazi [81] applied compressed sensing theory to show that the firing activity of different hippocampal regions was consistent with taking random projections to lower the dimensionality of a spike code and allow practical decoding performance substantially above the deterministic limit. Fagihi and Moustafa [48] applied mutual information as a measure to tune parameters of single neurons in a network model of schizophrenia, but assessed the resulting pattern separation functionality of the model using a form of Hamming distance. Vineyard et al [59] introduced a variety of loss-less coding strategies to estimate the maximal amount of information that could be encoded by a model dentate gyrus, and how this is increased by neurogenesis, but treated information maximisation as the ultimate goal rather than pattern separation as typically described. Severa et al [82] expanded on these ideas by describing a loss-less spike code for binary vectors that would increase sparsity and reduce correlations through an abstracted dentate-gyrus layer. In the cerebellum, Billings et al [58] studied how connectivity and cellular morphology support information throughput despite increased sparsity, concluding that these two features should be balanced. No previous study has, however, applied information as a direct measure of pattern separation efficacy in the dentate gyrus for arbitrary inputs.

We here introduce a set of techniques from information theory that allow pattern separation in the dentate gyrus to be rigorously quantified without the need to specify an accompanying pattern completion algorithm. We first confirm that arbitrarily removing structure from input spike train ensembles typically improves pattern separation as assessed by classical measures, but reduces the information content. We then show that mutual information [71], transfer entropy [83], and redundancy [84] can be used as measures of pattern separation performance and apply these to detailed compartmental models of individual granule cells and a small network. We further demonstrate how information estimation techniques [76, 85] can be adapted to spike train ensembles in order to reduce biases when data is scarce [86]. We demonstrate how our new measures might better predict pattern completion accuracy in an abstract auto-associative model. Finally, we demonstrate how structural changes of the dentate circuit associated with epilepsy [38, 40] affect pattern separation performance as assessed by our new measures and how they might apply to a larger network. We anticipate that information-theoretic measures of pattern separation will be a useful tool in disentangling the structure-function relationships of different cell types, how these relationships arise from learning and development, and how they are perturbed by pathology.

Sparsity typically increases standard measures of pattern separation, but decreases information transmission

Classical measures of pattern separation, such as orthogonalisation, decorrelation, or spike train distance, are effective at analysing the measured behaviour of a neuronal system and quantify the degree of difference between the inputs and outputs. In the dentate gyrus, outputs are typically encoded by sparser activity than the inputs and sparsity is a major contributor to pattern separation [21, 43, 44, 60, 82, 87, 88]. Increasing sparsity in a neural code, however, may reduce the information it carries about the afferent signal [58]. We here introduce a set of simple filters that arbitrarily increase the sparsity of an ensemble of spike trains and show both that classical measures of pattern separation can be highly parameter-dependent and that an arbitrary reduction in spike number typically leads to an increase in measured pattern separation performance.

Fig 1A (left) shows a raster of spike trains phase-locked to a periodic stimulus, and so displaying correlations both within and between spike trains (see Methods). Activity patterns consisting of ensembles of spike trains with different correlations within and between different trains are used throughout this study. Sparse trains are produced using thinning filters that remove spikes from the inputs in some way. Shown here is the independent uniform removal of individual spikes with a fixed probability p (Fig 1A, right), but we also consider three more structured filters: n-th pass—only every n-th spike in each train can pass, refractory—delete every spike within t seconds of a previous spike in the same train, and competitive—delete every spike within t seconds of another spike in any train in the ensemble (S1(B) Fig). These different filters pick out different relationships in the input spike train ensemble. The first filter destroys all structure in the ensemble as the deletion probability increases, the second and third filters maintain order or temporal structure within trains, and the fourth corresponds to competition between different trains in the ensemble.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Sparsifying (filtering) spike train ensembles leads to increased classical measures of pattern separation, but reduced information content.

A Demonstration of filtering a phase-locked spike train ensemble (left) by random spike deletion (right). The probabilities of spike deletion are p = 0.5, 0.75, 0.85, 0.95. B Standard measures of pattern separation as a function of discretisation bin size on a randomly filtered spike train with p = 0.5. Solid, dashed, and dotted lines refer respectively to strong, medium, and weak input similarities (phase-locked strengths of 0.75, 0.5, and 0.25, see Methods). From darkest to lightest, colours plot standard measures of pattern separation: orthogonalisation Υ_θ, scaling Υ_σ, decorrelation Υ_ρ, and Hamming distance Υ_η. Wasserstein distance Υ_δ does not rely on discretisation and is not plotted. All values are normalised for comparisons, raw values are plotted in S1(C)–S1(F) Fig. C Standard measures of pattern separation applied to filtered spike trains. Filtering methods and input similarities are as in B. The x-axis gives the filtering parameter in each case. All values are normalised for comparisons, raw values are plotted in S1(G)–S1(K) Fig. D Mutual information between input spike train ensembles and filtered spike train ensembles. Filtering methods and input similarities are as in B. Colours correspond to different neuronal codes (see Methods): from darkest to lightest instantaneous spatial, temporal, local rate, and ensemble rate. All values are normalised for comparisons, raw values are plotted in S2(A)–S2(D) Fig. E Mutual information as a function of classical pattern separation measures. Mutual information was maximised over all spiking codes and bin sizes for different input rates, correlation structures and strengths, and filter types and strengths. Clockwise from top left: orthogonalisation Υ_θ, decorrelation Υ_ρ, Wasserstein distance Υ_δ, and Hamming distance Υ_η. Shaded areas show one standard deviation. Raw scatter plots and densities are shown in S2(E)–S2(H) Fig.

https://doi.org/10.1371/journal.pcbi.1010706.g001

We next apply five metrics that are regularly used to assess pattern separation in the literature to spike train ensembles filtered in these ways. Overall this study compares three novel information-based and five existing measures of pattern separation. In all cases we use the letter Υ to refer to a measure of pattern separation; upper case latin subscripts denote the information-based measures and lower case greek subscripts denote classical non-information based measures. The classical measures are plotted in progressively lighter shades of red in Fig 1B. Orthogonalisation Υ_θ measures the reduction in pairwise cosine distance between normalised spike trains (Eqs 4 & 5), scaling Υ_σ measures the change in differences in the norms of the spike trains (Eqs 6 & 7), decorrelation Υ_ρ measures the reduction in pairwise correlations between spike trains (Eqs 8 & 9), Hamming distance Υ_η measures the increase in pairwise Hamming distance between spike train vectors (Eqs 10 & 11), and Wasserstein distance Υ_δ measures the increase in pairwise Wasserstein distance between spike trains (Eqs 12 & 13). These measures are fully described in Methods and summarised in Table 1. Orthogonalisation Υ_θ and scaling Υ_σ are complementary measures as orthogonalisation is computed on normalised vectors and scaling compares norms. The first four (Υ_θ, Υ_σ, Υ_ρ, and Υ_η) are computed on discretised spike trains, whereas the Wasserstein distance Υ_δ is parameter-free and computed on raw spike times. There are other ways to apply such metrics to pattern separation. The Hamming distance, for example, can also be applied to differences between entire ensemble patterns rather than individual spike trains in an ensemble [46, 49]; we take this form to keep the measures used as consistent as possible.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Table summarising classical pattern separation measures (top), and new information theoretic measures (bottom).

https://doi.org/10.1371/journal.pcbi.1010706.t001

Fig 1B plots the value of each measure (except the Wasserstein distance) normalised to its highest value (S1 Fig plots the unnormalised values in separate panels) for a random thinning with p = 0.5 as a function of the size of the discretisation bins used. In each case there is a monotonic relationship between the size of the bin and the value of the measure. This confirms the results of Madar et al [52] and means that it is very difficult to assign meaning to a given value of pattern separation computed by one of these methods as the value will typically depend on the choice of the binning parameter with no principled way to make this choice. It may be possible to pick approximate bin sizes based on biologically relevant timescales in some cases, potentially considering multiple different bins corresponding to multiple different timescales, but the precise width of the bins would remain a potential confound for any results.

Fig 1C plots the value of each measure normalised to its highest value (S1 Fig plots the unnormalised values in separate panels) as a function of spike deletion probability p. In all cases, each metric increases monotonically with the strength of the thinning filter and reaches a maximum when the output spike train ensembles are almost empty. This trend also holds for the other thinning filters (S1(G)–S1(K) Fig), with two exceptions: the Wasserstein distance measure Υ_δ takes a maximum at an intermediate value of the refractory filter (S1(K) Fig), and the decorrelation measure Υ_ρ takes a maximum at an intermediate value of the competitive filter, where spike trains inhibit each other (S1(J) Fig). As the Wasserstein distance is rate-invariant (normalised to the total number of spikes in each train), it appears that removing spikes that lie close to other spikes can lead to lower measured distances [53]. For the decorrelation measure, the competitive filter removes spikes close to others in all parts of the ensemble and so can lead to very sparse vectors that are actually more correlated in the chosen time windows.

The potential problem with such measures in isolation is illustrated clearly in Fig 1D, which plots the normalised mutual information between input and output spike train ensembles (S2 Fig plots the unnormalised values in separate panels). Different possible spiking codes (instantaneous spatial, temporal, local rate, and ensemble rate, see Methods) are plotted in different shades of blue. Although many of these codes also depend on binning spikes, it is possible to pick bin sizes in a principled way to maximise the information content. This is done separately for each condition. Pattern separation using the above classical measures increases as information is lost. This is a general trend and holds for different input correlation structures and strengths, and mean firing rates. Fig 1E supports this point by plotting the maximum mutual information over all spiking codes and bin sizes as a function of different classical measures for general spike train ensembles and filters. There is a universal negative relationship, meaning that increased classical measures of pattern separation typically imply an increased loss of information about the input.

Information theoretic measures of pattern separation

To avoid the above weaknesses of classical measures, we introduce two types of information theoretic measure of the ability of a neuronal system to separate patterns. The first type covers the relationship between inputs and outputs; giving a measure of the efficiency of the feedforward information transmission. The second type covers the relationships within the sets of inputs and outputs; giving a direct measure of the amount of information shared by an ensemble of spike trains which is more equivalent to the traditional definitions of pattern separation by orthogonalisation or decorrelation.

For the first type of measure, two related quantities are typically used to quantify the information throughput of a neural system: the symmetric mutual information I_X,Y between input X and output Y ([71], Eq 14), and the directed transfer entropy T_X→Y from input X to output Y ([83], Eq 15). Both have their advantages and a recent review found that the transfer entropy is typically less biased when data is sparse [89].

The second type of measure is linked to a quantification of the redundancy within a spike-train ensemble. The principle comes from the partial information decomposition of an ensemble into information that is encoded synergistically, independently, and redundantly by different components of that ensemble [90–92]. Removing redundancy between spike trains would distinguish them in terms of the information they convey and improve pattern separation. Whilst multiple measures of redundancy exist, we follow Williams and Beer [84] in defining it as the minimum mutual information R_X between different parts of a pattern X. Here this idea is applied as the minimum mutual information between each individual spike train and the rest of the ensemble (Eq 16).

The two types of informational measure can be used and combined to give robust measures of pattern separation. The information theoretic measures of pattern separation below are chosen to take a value of zero both in the limit of destroying all information between input and output, and when inputs have not been differentiated at all. To directly adapt the first type of measure to assess pattern separation, we take sparsity as a parameter-free proxy for pattern separation when combined with mutual information (Eqs 1 and 2). It would also be possible to take a traditional measure such as orthogonalisation or decorrelation and penalise this with information loss. The simple sparsity measure is preferred to these classical measures as it makes no assumptions on the pattern completion circuit, requires no discretisation of spike times, and has a natural relationship to the metabolic costs of spiking. The sparsity weighted measures therefore correspond to a more efficient coding of input patterns, likely using a higher-dimensional coding space [81, 93] where patterns will likely be more separated [58, 61, 88]. These measures can take negative values in the cases where the output contains more spikes than the input (ie the sparsity is ‘negative’), and can take similar values in cases where information throughput is high but sparsity is low and where information throughput is low but sparsity is high. There can also be value in considering the two components separately to see how far pattern separation performance is driven by maintenance of information throughput and how far by reduction in input similarity (see Results below).

To adapt the second type of information theoretic measure to assess pattern separation, we multiply the reduction in redundancy between the input ensemble and the output ensemble with the mutual information between the pair of ensembles, balancing feedforward information loss with reduced redundancy (Eq 3). All information-theoretic measures, as well as the classical measures discussed here, can be computed by our accompanying Matlab pattern separation toolbox. This toolbox combines both the calculation of the new metrics themselves, which depend on choices of neural code and numerical parameters such as discretisation binsize, and the optimisation algorithms we use to identify the codes and parameters that allow the maximum amount of information to be decoded.

Information theoretic measures of pattern separation and filtered spike ensembles.

Fig 2 shows the application of these measures to filtered spike train ensembles as in Fig 1. Fig 2A shows the sparsity weighted mutual information Υ_M, Fig 2B the sparsity weighted transfer entropy Υ_T, and Fig 2C the relative redundancy reduction Υ_R. In all cases there is a single peak in each measure at an intermediate value of the thinning filters, and this also holds for the nth-pass, refractory, and competitive filters (S3 Fig). As before, different assumed spiking codes lead to different relationships between sparsity and information transmission (different shades of blue in panels A to C), but the principled approach is to take the codes that produce the highest information throughput. For Υ_M and Υ_T, the measures are relatively independent of the strength of the phase-locking within the input patterns (see S3 Fig for unnormalised values). Υ_R, however, which accounts for the reduction in redundancy between input and output patterns, is strongly dependent on the strength of input similarity (here the strength of the phase-locking), with stronger similarities leading to higher measures of pattern separation at all thinning filter values. Fig 2 plots the informational measures for a number of different spike codings, with parameters chosen to maximise each amount of information for each code. The optimal parameters, for both the inputs and outputs, differ for each datapoint, but the overall trend in informational terms is preserved. In general, we will also optimise over a number of different possible encodings; the goal is to identify the maximum amount of information that could possibly be contained in a given signal and so correctly penalise the absolute loss of information, however measured. This does not require an assumption that the spiking codes used by the entorhinal cortex maximise information efficiency, merely that it might be possible for a circuit to decode all of the information in a spike train ensemble.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Information theoretic measures of pattern separation penalise pattern destruction (information loss).

A Normalised sparsity weighted mutual information Υ_M applied to randomly filtered spike train ensembles. Solid, dashed, and dotted lines refer respectively to strong, medium, and weak input similarities (see Methods). Colours correspond to different neuronal codes (see Methods): from darkest to lightest instantaneous spatial, temporal, local rate, and ensemble rate. All values are normalised for comparisons, raw values are plotted in S3(A) Fig. B Normalised sparsity weighted transfer entropy Υ_T applied to filtered spike train ensembles. Colours as in panel A. All values are normalised for comparisons, raw values are plotted in S3(B) Fig. C Normalised relative redundancy reduction Υ_R applied to filtered spike train ensembles. Colours as in panel A. All values are normalised for comparisons, raw values are plotted in S3(C) Fig.

https://doi.org/10.1371/journal.pcbi.1010706.g002

Υ_M is a simple to understand and easy to apply proxy for pattern separation; it can be replaced by Υ_T if desired or if data is sparse (but see below for a procedure to estimate the mutual information in this case). Υ_R provides a useful complementary measure with more sensitivity to the strength and structure of input correlations.

Single cell pattern separation

Whilst full pattern separation likely relies on circuit structure [5, 45], features of single cells embedded in their networks can contribute to the separation of patterns in time [44]. If a single neuron is sequentially presented with similar input patterns and maps these to less similar output patterns, then measures of pattern separation can be computed on the combined input and output ensembles. This would be equivalent to a population of independent neurons simultaneously receiving different elements of the input ensemble and simultaneously producing different elements of the output ensemble.

To demonstrate the application of these new measures to simulated data from physiologically realistic single cell models, we study the effects of changing parameters on the ability of a detailed compartmental model of a single mouse granule cell [94] to separate patterns (Fig 3A). The cell receives informative synaptic input from the lateral perforant path (synaptic contacts shown as large yellow markers in Fig 3A), and background input across the rest of its dendritic tree (brown markers in Fig 3A). Inputs are presented as repeated structured patterns (rasters on the left of Fig 3A). Specifically, each spike train in the ensemble is presented sequentially and the output of the neuron is recorded. These outputs are combined into an output ensemble. Perforant path inputs to granule cells undergo stochastic short-term plasticity (see Methods for details), and varying the timescales of depression and facilitation alters both the sparsity-weighted mutual information Υ_M and relative redundancy reduction Υ_R (Fig 3B, top two panels). Both measures show a consistent and robust peak in pattern separation at timescales of ∼ 500ms for depression and ∼ 300ms for facilitation, indicating an optimal parameter value for separation of the input patterns. The experimentally estimated values for these timescales at lateral perforant path synapses in Madar et al [44] are ∼ 500ms for depression and ∼ 9ms for facilitation. Similarly, the public Hippocampome.org dataset [95] gives average recorded timescales of 563.9ms for depression and 8.33ms for facilitation at the time of writing. The sparsity weighted mutual information measure Υ_M is relatively unaffected by changes in the similarities of the input patterns compared to the relative redundancy reduction measure Υ_R, which shows higher values when input similarities are stronger. The peaks of the curves are not affected by the strength of the input similarity when changing the depression timescales, but stronger similarities seem to slightly favour slower facilitation, with a rightwards shift in both the Υ_M and Υ_R curves. Using classical measures of pattern separation (see S4 Fig) does not reveal a consistent peak in performance, with most measures being either flat or monotonic across the range of timescales and usually giving results that depend on the size of discretised bin used.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Information theoretic measures applied to a single cell provide consistent and robust assessment of pattern separation.

A Left panels: Example input spiking rasters. Right panel: Dentate gyrus granule cell morphology. Informative synaptic contacts are shown by yellow and background synaptic contacts by brown markers. Example output voltage traces from the granule cell soma are shown in S4(A) Fig. B. Sparsity weighted mutual information Υ_M (left) and relative redundancy reduction Υ_R (right) for the granule cell model as a function of various physiological parameters. Spiking codes are chosen separately for the inputs and outputs to maximise information. From top to bottom: timescale of synaptic depression, timescale of synaptic facilitation, and spatial heterogeneity in ion channel densities (see Methods). Each input is presented 8 times. Solid lines represent the mean over 10 repetitions with different inputs and the shaded areas show one standard deviation above and below the mean. In general, input and output measures were close to the mean. Blue shows a weak input similarity (phase-locked correlation strength of 0.25), and grey a strong input similarity (phase-locked correlation strength of 0.75). Spike traces are two minutes long and consist of phase-locked inputs with a phase rate of 0.6Hz and a spiking rate of 5Hz. C Components of pattern separation as a function of the timescale of synaptic depression. From top to bottom: sparsity (m_X − n_Y)/m_X, mutual information I_X,Y, and redundancy reduction R_X − R_Y (see Eqs 1 to 3).

https://doi.org/10.1371/journal.pcbi.1010706.g003

In contrast to synaptic properties, the pattern separation performance of the cell is relatively unaffected by increasing intrinsic heterogeneity. The bottom row of Fig 3B shows pattern separation performance as a function of spatial ion channel heterogeneity. This is implemented by randomising the densities of the model’s ion channels around their mean in each 1μm section of the neuron with a given coefficient of variation (see Methods). The lines for both measures are relatively flat, with no substantial impact of spatial heterogeneity on either Υ_M or Υ_R. Changing the spatial heterogeneity in this way does not, on average, affect the excitability of the cell, but does alter the relative impacts of different synaptic contacts. In this model this change is insufficient to significantly alter the pattern separation performance.

After assessing the overall pattern separation performance, it is possible to investigate how the different components of our new measures may contribute. Fig 3C plots how sparsity, pure mutual information, and pure redundancy reduction change as a function of the timescale of synaptic depression. Sparsity is a purely increasing function, mutual information is a purely decreasing function, and redundancy reduction itself takes a peak for relatively slow vesicle recovery. Although such an analysis is helpful in identifying how a system is performing different aspects of pattern separation, it is not typically possible to identify the parameters that might give peak performance as shown by the combined measures Υ_M and Υ_R.

Synapses with stochasticity and short-term plasticity are believed to contribute to pattern separation [44], gain modulation [96, 97], and energetically efficient information throughput [98]. The new techniques introduced here show how altering the timescales of this plasticity quantitatively changes the pattern separation performance of a single granule cell, and how experimentally measured timescales appear to be highly effective for pattern separation by balancing reduction of redundancy with loss of mutual information.

Estimators for mutual information and performance on sparse data

A limitation of some informational measures is that they can require large amounts of data to produce accurate estimates of information content [86, 89]. This issue has been addressed by the development of estimators for informational quantities based on some measure of the distances between spike trains. In the Methods section we describe how to adapt the modified Kozachenko-Leonenko estimator [85] described in Houghton [76] to account for spike train ensembles and measurement of redundancy (Eqs 17 to 19). To apply the estimate consistently we also sought to remove a key parameter of the original implementation, the number of segments a spike train is divided into. We found that this parameter was unnecessary by adapting the technique Strong et al [75] introduced to estimate the limiting mutual information of infinitely long spike trains. Fig 4A (left panel) shows that, for large numbers of segments (see Methods), the mutual information estimate is a linear function of the number of segments. By extrapolating back to the case of 0 segments, which must therefore be infinitely long, we find an estimate of the limiting mutual information of a spike train. A similar technique can be used for redundancy estimates (Fig 4B (left panel)).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Mutual information and redundancy can be estimated in situations of limited data.

A Left: Example point Kozachenko-Leonenko (KL) estimates (triangles) of mutual information (MI) for a spike train as a function of the number of segments (see Methods) and the extrapolation to infinite length (solid line and arrow). Dashed lines show the 95% confidence interval of the estimate. Right: Mean and standard deviation (shaded areas) in mutual information estimates for single-trial data as a function of spike train length. Blue shows the exact calculation using a temporal code and purple the Kozachenko-Leonenko estimate. B Left: Example point Kozachenko-Leonenko (KL) estimates (triangles) of redundancy (Red.) for a spike train as a function of the number of segments and the extrapolation to infinite length (solid lines and arrows). Dark blue shows the input ensemble (in) and light blue the output ensemble (out). Right: Mean and standard deviation (shaded areas) in redundancy reduction (RR) estimates for single-trial data as a function of spike train length. Colours as in panel A.

https://doi.org/10.1371/journal.pcbi.1010706.g004

By replacing the exact quantities in Eqs 1 and 3 with their estimators, this leads to estimators for sparsity weighted mutual information and relative redundancy reduction denoted and respectively. These estimators have advantages when data is sparse. Fig 4 (right panels) plots the distribution of estimates of mutual information and redundancy reduction using both the explicit temporal code (blue) and the Kozachenko-Leonenko estimators (purple) when only a single short trace over five spike trains is available. The standard deviation in mutual information estimates using the estimator is consistently smaller than the explicit calculation. Even when experimental data is sparse therefore, there are reliable ways to estimate pattern separation efficacy using our new information theoretic techniques. In specific circumstances, with relevant preliminary data or good estimates of circuit features such as input patterns and connectivity, it would be possible to use simulations such as this to constrain the length of the experiments needed to produce accurate exact values of Υ_M and Υ_R.

The estimators and can be computed by our accompanying Matlab pattern separation toolbox.

Information theoretic measures can predict pattern completion

A major strength of the information theoretic approach to measuring pattern separation is that it allows one to remain agnostic about the nature of any mechanisms of pattern completion in downstream circuits. Nevertheless, it is possible to demonstrate that our new measures have value in quantifying the ability of an abstract pattern completion circuit to accurately recover inputs. The Hopfield network is a classical autoassociative network that is able to reconstruct complete patterns from partial inputs once it has been trained [99]. Hopfield networks suffer from catastrophic interference when learning to reproduce multiple patterns, and the extent of this interference typically depends on the degree of correlation between the trained patterns [100].

To evaluate how well information theoretic measures of pattern separation might predict pattern completion in this abstract model, we took the Kuzushiji-49 dataset of 49 handwritten Japanese Hiragana characters that is widely-used as a benchmark for computer vision algorithms [101]. The digits are encoded as 28 pixel by 28 pixel grayscale images, or as vectors of length 784. This means that the digit is represented by a population code. As handwritten digits, the elements of the dataset contain spatial correlations that impair the storage capacity of a Hopfield network. To train the network, the patterns are binarised by thresholding the grayscale values, and an exemplar of each class is chosen at random (S5(B) Fig). A Hopfield network with 784 nodes is then trained on the set of 49 exemplar patterns using the pseudo-inverse learning rule [102]. To assess the recall accuracy of the trained network, it is presented with each pattern in the full dataset and left to converge. The recalled class is then taken to be the class of the exemplar that best matches (has the greatest dot product with) the converged pattern. The initial recall accuracy is 16.83%, reflecting the problems caused by the correlated patterns.

Whilst it is possible to apply a basis transformation to the dataset to achieve high pattern completion performance, noise can also be used to separate the patterns. Noise is applied by randomising a subset of the pixels at fixed locations in each class (Fig 5A). This provides a noisy transformation on the input patterns; pattern separation metrics can be computed on this transformation (see Methods). Increasing the number of randomised pixels initially increases the performance of the network, whilst reducing correlations (Fig 5B). Beyond a certain point, however, the noise dominates and completion accuracy falls quickly towards a chance level. This is reflected in the relative redundancy reduction Υ_R of the noisy pattern transformation, which also takes a peak at an intermediate level of noise. As the patterns are not encoded by spikes, the sparsity-weighted mutual information Υ_M is not appropriate for this system. Directly comparing Υ_ρ and Υ_R with the pattern completion accuracy (Fig 5C and 5D) shows that, in this abstract model, Υ_R can be a useful predictor of subsequent pattern completion performance but Υ_ρ has poor predictive power.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Information theoretic measures can predict pattern completion performance.

A Example binary patterns from the Kuzushiji-49 dataset (top to bottom) with increasing levels of pixel noise (left to right: 0, 200, 400, and 600 pixels). B Top: Completion accuracy of a Hopfield network as a function of noise strength. Middle: Correlations between different classes of pattern as a function of noise strength. Bottom: Relative redundancy reduction (Υ_R) between classes of patterns as a function of noise strength. Solid line shows the mean over 1000 repetitions and the shaded area shows the standard error. C Completion accuracy of a Hopfield network against decorrelation Υ_ρ. D Accuracy against Υ_R.

https://doi.org/10.1371/journal.pcbi.1010706.g005

This abstract model has a number of important qualitative differences from the real hippocampus. In particular, patterns are purely binary and presented instantaneously and separately to the network, with no background interference from other processes. Further, training and recall patterns pass through exactly the same transformation. In reality, hippocampal pattern completion circuits receive two streams of input, through the dentate gyrus and directly from the entorhinal cortex. These two streams likely have different roles in training and recall [1, 5, 79]. In addition, the model only covers one type of (population) code, and one type of pattern completion circuit. Nevertheless, the model illustrates how reducing redundancy whilst maintaining mutual information can be beneficial for storage and recall in a recurrent network and provides further motivation for our new metrics.

Pattern separation in a network

Most studies of the dentate gyrus find that effective pattern separation in the dentate gyrus is primarily a network phenomenon [5, 61] produced by effects such as unreliable expansion to multiple principal cells, conditional reinforcement through lateral excitation, and competitive inhibition by interneurons [45, 61]. In addition, heterogeneities in the response properties of individual granule cells are likely to contribute [103]. We next show the applicability of our information theoretic pattern separation measures to the output of a simple microcircuit model of the mouse dentate gyrus. The full details are given in Methods, but overall we consider a population of 40 mature granule cells, 10 adult-born granule cells, 5 mossy cells, and one pyramidal basket interneuron (Fig 6A and 6B). The principal cells receive informative inputs in the outer molecular layer undergoing stochastic short-term plasticity as above, 5 groups are formed by sets of 8 mature and 2 adult-born cells receiving the same informative presynaptic spikes, but with independent stochastic synaptic dynamics. The output is taken to be the ensemble of all granule cell spike trains.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Pattern separation in a network and under pathology.

A Example of dentate gyrus microcircuit. Granule cells are in shades of blue, mossy cells are in shades of brown, and the pyramidal basket cell is in red. B Example rasters of input patterns (left) and driven microcircuit activity (right). Vertical rows show spiking activity of a single cell and colours correspond to panel A. Cells are grouped into sets receiving the same inputs and sharing preferential lateral connections (see Methods). C. Sparsity weighted mutual information Υ_M (top) and relative redundancy reduction Υ_R (bottom) for the mature (left) and adult-born (right) granule cell populations as a function of input phase-locked correlation strength. Solid lines represent the mean over 10 repetitions and the shaded areas show one standard deviation above and below the mean. Blue shows the response to input ensembles with a spiking rate of 5Hz and grey to input ensembles at 10Hz. Spike traces are two minutes long and consist of phase-locked inputs with a phase rate of 0.6Hz. D. Sparsity weighted mutual information Υ_M (top) and relative redundancy reduction Υ_R (bottom) for the mature (left) and adult-born (right) granule cell populations as a function of a scaling parameter for the informative lateral perforant path synapses. Solid lines represent the mean over 10 repetitions and the shaded areas show one standard deviation above and below the mean. Blue shows a weak input similarity (phase-locked correlation strength of 0.25), and grey a strong input similarity (phase-locked correlation strength of 0.75). Spike traces are two minutes long and consist of phase-locked inputs with a phase rate of 0.6Hz and a spiking rate of 5Hz. E Simulations of epilepsy-related changes in synaptic input weights (horizontal axis) and Kir2 channel density (vertical axis). Sparsity weighted mutual information Υ_M as a joint function of synaptic and Kir2 expression scaling parameters. The heatmap shows the average over ten repetitions. Spike traces are two minutes long and consist of phase-locked inputs with a strength of 0.75, a phase rate of 0.6Hz, and a spiking rate of 5Hz. The relative redundancy reduction Υ_R is plotted in S5(B) Fig. F Components of pattern separation as a joint function of synaptic and Kir2 expression scaling parameters. From left to right: sparsity (m_X − n_Y)/m_X, mutual information I_X,Y, and redundancy reduction R_X − R_Y (see Eqs 1 to 3).

https://doi.org/10.1371/journal.pcbi.1010706.g006

We first apply our measures to evaluate the relative pattern separation performance of mature and adult-born granule cells in the model. Fig 6C plots the sparsity weighted mutual information Υ_M and relative redundancy reduction Υ_R for both populations as a function of the strength of input similarities. All informational quantities are maximised over the relevant numerical parameters and the set of possible spike codes we consider. By both measures the pattern separation efficacy of the mature population is higher than that of the adult-born population in absolute terms, with a similar trend across both populations and measures in response to increasing input similarities. The increase in Υ_R with input similarity in the mature population is due to the fact that higher levels of input similarity imply higher absolute levels of input redundancy. The adult-born cells do not display any significant reduction in input redundancies at any similarity level. The reason for this appears to be that the adult-born cells in this model are more intrinsically excitable than the mature cells [94], and the modelled perforant path inputs are not scaled down to match this. There is evidence that excitatory inputs to adult-born cells in the real dentate gyrus are weaker to balance their higher intrinsic excitability and maintain sparse activity [104, 105]. Ultimately, the scaling of afferent synapses onto adult-born granule cells [106] and the extent of their integration into dentate gyrus microcircuits [12] will determine their contribution to pattern separation in the brain.

Pattern separation under pathology

Next we provide a proof of principle and demonstrate the usefulness of our information theoretical measures for assessing the function of the dentate gyrus under pathological conditions, when changes to input codes and pattern completion mechanisms are potentially unknown. For this purpose, we apply our new measures to estimate the performance of the above network under the types of changes associated with epilepsy. Epilepsy causes structural changes in the dentate gyrus that increase granule cell activity and reduce behavioural discrimination performance [43]. A notable structural change is a significant increase in the sizes of axonal boutons and dendritic spine heads at perforant path synapses, and an associated increase in synaptic strength [40]. This is likely to hinder pattern separation by increasing the reliability of input responses by each principal cell [39]. In agreement with this expectation, scaling up the informative synapses to the granule cells in the above networks shows a decrease in pattern separation performance by both Υ_M and Υ_R (Fig 6D).

Another change in the epileptic dentate gyrus, is an increase in the expression of inward rectifying potassium (Kir2) channels in granule cells [38]. This mechanism increases the conductance of granule cells and so reduces their excitability, potentially compensating for increased afferent synaptic strength. Measuring Υ_M as a function of scaling parameters for both the Kir2 channel densities and the informative synaptic strengths, reveals a region where pattern separation performance could be maintained by upregulating Kir2 expression to counteract increased synaptic weights (Fig 6E). To see exactly how this occurs, Fig 6F plots how sparsity, pure mutual information, and pure redundancy reduction change as a joint function of the synaptic scale and Kir2 conductance scale. Sparsity and redundancy reduction decrease with synaptic strength, but increase with Kir2 conductance. Mutual information has a largely inverse relationship to the other two measures, but not so much as to cancel out the region of higher pattern separation seen in Fig 6E. These results show that the new information theoretic measures of pattern separation produce reliable and intuitive results when applied to complex compartmental models that seek to simulate real systems.

Pattern separation in a large network

The pattern separation measures defined here are not limited to small networks and can be scaled up to larger systems. In Fig 7 we simulate networks of 50, 000 and 100, 000 granule cells receiving 5, 000 or 10, 000 two minute long inputs that are generated from a grid cell-type firing pattern (Fig 7A). The granule cells are simplified to point neurons with adaptive exponential integrate-and-fire spiking properties fitted to the compartmental models above [107, 108]. In this model it is possible to explore a number of network features at larger scale and directly compare the performance of the sparsity-weighted mutual information Υ_M against a classical measure, the Hamming distance Υ_η, for networks of very large size.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. New pattern separation measures can be applied to larger network models.

A Example of inputs to the large network model. Top. Example path in space that generates grid cell-like firing. Colour gradient indicates time over 20s. Bottom: Input firing rasters of 50 neurons over the shown path for 20s. Colours match the time above. B Pattern separation as measured by Hamming distance Υ_η (top) and sparsity-weighted MI Υ_M (bottom) as a function of the maximum firing rate of input cells. Dark lines correspond to 5, 000 inputs to 50, 000 principal cells, and light lines to 10, 000 inputs to 100, 000 principal cells. Error bars show one standard deviation over 24 repetitions. C Pattern separation as a function of the number of principal cells innervated by each of the 100 (50k network) or 200 (100k network) inhibitory basket cells. D Pattern separation as a function of the synaptic strength of the inhibitory basket neurons. A scale of 1 corresponds to 1nS. E Pattern separation as a function of the proportion of principal neurons that are adult-born. Other parameters are as described in Methods and colours and markers are as described in B.

https://doi.org/10.1371/journal.pcbi.1010706.g007

We varied the maximum firing rate of the inputs, and found an increase in Υ_M with firing rate (Fig 7B, using a population rate code), but not consistent changes in Υ_η (all brown lines in Fig 7) at this scale. When varying the number of principal neurons innervated by each of the 100 or 200 inhibitory basket cells we found an intermediate peak in Υ_M (Fig 7C), matching the idea that a balanced level of recurrent inhibition is best for pattern separation. A similar result was found for up- and -downscaling the strength of the inhibitory synapses from the basket cells to each of their 500 efferent granule cells 7D). Finally, increasing the proportion of adult-born granule cells led to a linear decrease in pattern separation performance 7E). This result is consistent with our detailed compartmental simulations in Fig 6C, but again may change if the integration of the adult-born cells into the feedforward and lateral connectivity of the network is altered according to experimental data on their synapses as the adult-born cells provide another source of network heterogeneity [12, 106]. There is also likely to be some interaction between the proportion of more excitable adult-born cells and the optimal level of recurrent inhibition in Fig 7C and 7D.

Interestingly, doubling the size of the network from 50, 000 to 100, 000 principal cells consistently leads to a slight increase in Υ_M for all network parameters (dark blue lines against light blue lines in Fig 7). This is driven by a slight increase in the mutual information of the ensemble codes with size.

Standard metrics of pattern separation

A number of measures have been used to quantify pattern separation in neuronal data. We here use five existing measures of spike train dissimilarity for comparison with our new information theoretic measures. These are designated with Υ and a lower-case greek subscript to differentiate them from the upper-case latin subscripts used for the information theoretic measures. In general, we consider metrics of pattern separation as the ratio of output similarity to input similarity. This minimises input similarity as an additional source of variability and keeps the measures consistent, but is different from the approach of studies such as Madar et al [52] where the difference between input and output similarities is used. Other studies consider the input and output values separately [49], their decreased overlap [14], or the values integrated over a range of different inputs [45]. The latter approach removes the influence of input similarity on the measure, but requires more simulation time. Studies such as Yim et al [39] plot both input and output similarities, whilst fitting ratios as here in order to quantify changes in pattern separation performance. Such an approach retains the benefit of presenting both input and output similarities separately for direct visual comparison, a common feature in pattern separation studies, but also provides a single metric to directly measure changes in performance. For all measures that require discretisation, a bin size of 10ms is used unless otherwise stated.

Mutual information

The mutual information I_X,Y between two random variables X and Y quantifies the reduction in uncertainty about the value of one variable that knowing the value of the other would provide [71]. It takes a minimum of 0 for independent random variables and a maximum of the entropy of either random variable if the relationship is entirely deterministic. I(X;Y) is defined as (14) where p_(X,Y) is the joint distribution of X and Y and p_X and p_Y are their respective marginals.

For neural data the distributions p_(X,Y), p_X, and p_Y depend on the choice of neuronal coding strategy (see ‘Neuronal codes’ below) and discretisation. The coding strategy and discretisation should in principle be chosen to maximise I in each case.

Transfer entropy

The transfer entropy T_X→Y [83] from one random variable X to another Y is the mutual information between Y at the current time T and the history of X conditioned on the history of Y: (15) where H(X|Y) is the conditional entropy of X given Y. As for the mutual information, the distributions depend on the choice of neuronal coding strategy (see ‘Neuronal codes’ below) and discretisation, but these can be chosen to maximise the measured entropy. Unlike mutual information, transfer entropy is directional and asymmetric between inputs and outputs. In many circumstances, the transfer entropy requires less data to produce an accurate estimate than the mutual information [86, 89].

Redundancy

Redundancy means that multiple parts of a signal may encode the same information [78, 84, 92, 135]. To use this idea to asses pattern separation, we adapt the partial information decomposition of Williams and Beer [84] to spike trains in an ensemble. In particular, if X is a spike train ensemble made up of trains {x₁, x₂, …, x_n} we consider the redundancy of X to be the minimum mutual information between any individual train x_i and the remainder of the ensemble X\x_i (16) Again, the probability distributions depend on the choice of neuronal code. Redundancy can be applied to both the input and output ensembles, and these are combined to give the pattern separation measure redundancy ratio Υ_R (Eq 3).

Neuronal codes

The joint and marginal distributions over the spike trains depend on which features are considered. We use the common terminology of ‘words’ to refer to the significant patterns that make up the sample spaces over which information is calculated; ‘words’ can be formed from different combinations of ‘letters’. All distributions are computed empirically from the data, without assuming a specific functional form. It is not necessary for input and output spike train ensembles to use the same coding strategy, so informational measures are maximised over all possible input-output pairs.

Estimating mutual information and redundancy

Mutual information can also be estimated using a variety of standard techniques [136]. We demonstrate the application of such techniques using the modified Kozachenko-Leonenko estimator [85] described in Houghton [76]. This estimator exploits the proximity structure of spike trains to reduce the amount of data necessary to reliably estimate the mutual information between them. Briefly, the input and output spike train ensembles are each divided into N periods of equal size. The pairwise spike train distances between each period of each spike train are computed using the Wasserstein metric δ (Eq 12) and a euclidean norm is taken over the trains in the ensemble. This produces a set of pairwise distances between periods of the input and output ensembles. A biased estimator of the mutual information between input and output ensembles in terms of an integer smoothing parameter 1 ≤ h < N is given by (17) where #(C_h(u_i, v_i)) counts the number of pairs (u_j, v_j) such that both δ(u_i, u_j) < Δ_h(u_i) and δ(v_i, v_j) < Δ_h(v_i), where Δ_h(u_i) is the distance of u_i to its h-th nearest neighbour amongst all segments u. The bias can estimated as (18) We follow Houghton [76] in choosing the smoothing parameter h to maximise the unbiased estimator , specifically. (19) The original implementation of this technique assumed a fixed period of division, leading to a fixed value of N if the total length of the spike train ensemble is constant. We relax this assumption by adapting the technique in Strong et al [75] to estimate the limiting mutual information for infinitely long spike trains. Briefly, is calculated for different values of N and values are considered reliable if the optimal smoothing parameter h is much lower than its maximum possible value of N − 1 (S5(A) Fig, top panels). Estimates in this region are a linear function of the number of periods N. The linear fit is then extrapolated back to a value of N = 0 to produce a consistent estimate of the total mutual information between the spike train ensembles (Fig 4, right panels, and S5(A) Fig, bottom panels).

The procedure to estimate the redundancy between trains in an ensemble is similar. As in Eq 16, we compute the minimum estimated mutual information between each train in the ensemble and the remainder taken together. This is extrapolated to the limiting mutual information as above before the minima are taken.

Input spike trains

Ensembles of input spike trains are generated using a number of different techniques to produce different correlation structures across and within different trains. The different ensembles possess positive and negative correlations both across and within distinct trains [137, 138]

• Phase-locked. Phase-locked ensembles are generated using an inhomogeneous Poisson process with the instantaneous firing rate varying sinusoidally in time and constant across spike trains. This generates sets of spike trains that are periodic in time and positively correlated between different trains. Key parameters are the strength of the phase-locking, the relative amplitude of the sinusoid about the mean single neuron firing rate, and the period of the sinusoid.
• Auto-correlated. Spike trains are generated independently using a renewal process with gamma-distributed interspike intervals. The shape parameter α of the gamma distribution governs the autocorrelation of the trains. Shape parameters less than one produce bursty trains, a shape parameter of one is equivalent to an uncorrelated Poisson process, shape parameters greater than one produce periodic trains. The initial spike time in each train is generated by a Poisson process to eliminate cross-correlations.
• Cross-correlated. Spike trains are produced by uniformly filtering a master Poisson process with rate equal to the desired rate multiplied by the product of the number of trains in the ensemble and the correlation strength.

Examples are plotted in S1(A) Fig.

Granule cell models

We conduct our simulations using the morphologically robust compartmental models of mouse dentate gyrus granule cells introduced by Beining et al [94]. In brief, these models distribute the ion channels reported in the literature over morphologies based on those reconstructed by Schmidt-Hieber et al [139]. The ion channels are section specific to the soma, axon hillock, axon, granule cell layer dendrites, inner molecular layer dendrites, middle molecular layer dendrites, and outer molecular layer dendrites. Full details can be found in the original paper.

Synaptic inputs from the perforant path undergo short-term plasticity. This is modelled using the quantal version of the Tsodyks-Markram model [140], where individual vesicles undergo probablistic release and replenishment. The standard parameters are adapted from those fitted in Madar et al [44] to the lateral perforant path. The initial release probability is 0.325, there are 40 vesicle release sites per synapse, the timescale of recovery from depression is 500ms, and the timescale of facilitation is 9ms. The excitatory postsynaptic conductance change induced by a single vesicle is 0.3nS. The synapse is formed by four anatomical contacts in the outer molecular layer (see Fig 3A). In addition, each cell receives background noise from weaker synapses, these are distributed randomly with densities of 0.05, 0.1, 0.2, and 0.1 contacts per μm in the granule cell layer, inner molecular layer, middle molecular layer, and outer molecular layer respectively. Background synapses undergo independent Poisson activation with a rate of 0.5Hz and a strength of 0.01nS. All excitatory synapses are modelled as a difference of exponentials, with a rise timescale of 0.2ms, a decay timescale of 2.5ms, and a reversal potential of 0mV.

The original model assumes that ion channel conductances are constant within each section of the tree (granule cell layer, inner molecular layer, etc). For the bottom panels of Fig 3B we vary the conductance randomly within each section. Conductances in each 1μm segment are randomly drawn from a non-negative gamma distribution with mean equal to the fitted value and a coefficient of variation (ratio of standard deviation to mean) that is used to vary the spatial heterogeniety as seen in the figure. Specifically, if the mean conductance of an ion channel in a layer is μ, and the desired coefficient of variation (heterogeneity) is c > 0, then the conductance of that ion channel in each 1μm section is drawn independently from (20) where Γ(z) is the gamma function.

A typical set of input spike patterns and corresponding output voltage traces are plotted in S4(A) Fig.

Pattern completion

In Fig 5 we give a proof-of-principle demonstration as to how our novel measures of pattern separation would affect pattern completion in an abstract auto-associative Hopfield network [99]. The network consists of 784 recurrently-connected artificial binary neurons that each have a state s_i ∈ {−1, 1}. The network is initiated with a given pattern and evolves asynchronously, so that individual neuron states are updated randomly until the network has converged to a final state. Updates occur using the rule (21) where is the state of neuron j immediately before the update, is the state of neuron i immediately after the update, and w_ij is the connection weight from neuron j to neuron i. Weights are calculated for an exemplar pattern of each of the 49 classes (S5(B) Fig) using the pseudo-inverse training rule [102]: (22) where ν and μ index the 49 patterns, is the i-th element of training pattern ν, and Q_νμ is the overlap (23)

The noisy transformation is applied to patterns by selecting m pixels uniformly randomly. These pixels then take a uniform random value in {−1, 1}. For each transformation, the locations of the randomised pixels are fixed, but their values are generated independently for each pattern. The correlation ρ between two different pattern classes is calculated as the average between all patterns in the classes. The mutual information for each class is calculated between the example pattern used to train the network and the set of all other patterns in the class. The redundancies are calculated between the different classes.

Network models of detailed neurons

In Fig 5 we consider a small network of excitatory and inhibitory cells from the the mouse dentate gyrus. The network is randomly generated for each repetition, with potentially different morphologies and connectivites. The principal cells are 40 mature and 10 adult-born granule cells adapted from the single cell model described above. Heterogeneity is included by using the range of available morphologies introduced in Beining et al [94] and varying the channel densities with a coefficient of variation of 0.1 for the mature and 0.2 for the adult-born cells. 5 mossy cells and 1 inhibitory pyramidal basket cell are modelled as integrate-and-fire units with a membrane time constant of 15ms.

Connectivity for the microcircuit broadly follows the principles described in Hainmueller and Bartos [5] and employs motifs believed to support pattern separation as discussed by Cayco-Gajic and Silver [61].

• Perforant path to granule cells. These synapses are as described in the single cell case above, with informative synapses innervating the outer molecular layer and displaying quantal short-term plasticity. Background synapses randomly innervate the rest of the dendritic tree, with a fixed independent likelihood based on dendritic length. There are no differences in synaptic structure or parameters between adult-born and mature granule cells. Groups are formed by sets of 8 mature and 2 adult-born cells receiving the same informative presynaptic spikes, but with independent stochastic synaptic dynamics. This provides unreliable expansion to a larger coding space.
• Granule cells to mossy cells. Granule cells form excitatory connections to mossy cells. Each group of granule cells is preferentially recurrently connected to a single mossy cell, with probabilities of cells inside the group contacting the mossy cell being 0.9 each, and 0.025 for innervating each of the other mossy cells. The mossy cells act as a second threshold.
• Mossy cells to granule cells. Mossy cells form lateral excitatory connections to granule cells. Each mossy cell is preferentially connected to a group of granule cells, with probability 0.9 of contacting each cell inside the group and 0.01 of contacting each granule cell outside the group. Synapses are located at the soma of the granule cell and have a conductance of 0.5nS.
• Granule cells to pyramidal basket cells. Mature granule cells have the potential to form excitatory connections to the pyramidal basket cell and do so with probability 0.5. Adult-born granule cells do not form these connections. This motif provides competitive inhibition between groups of principal cells.
• Pyramidal basket cells to granule cells. The pyramidal basket cell forms lateral inhibitory connections to all granule cells, including the adult-born population. Synapses are located at the granule cell somata and have a conductance of 1nS, a double exponential timecourse with rise and decay timescales of 0.5ms and 5ms respectively, and a reversal potential of −80mV.

Further connections, such as direct granule cell recurrency, which is observed in primates but not healthy rodents [5], are not included.

Network models of point neurons

In Fig 7 we expand our detailed model to consider much larger networks and so reduce the fidelity of our simulations. The compartmental models of mature and adult-born granule cells are replaced with single-compartment adaptive exponential integrate-and-fire neurons [107, 108]. The voltage V of each neuron evolves as (24) where τ is the neuron time constant, E_v is the reversal potential, Δ_T is the sharpness of action potential initiation, θ_T is the ‘threshold’, R is the somatic input resistance, I(t) is the input current, and w(t) is an adaptation variable governed by (25) where τ_w is the adaptation time constant, a is the adaptation coupling strength, b is the increase in w caused by a single spike, and δ is a Dirac delta function. The voltage in Eq 24 goes to infinity in finite time once it exceeds θ_T, so a spike is recorded once the voltage exceeds 0mV and the voltage is reset to E_v.

To match the responses of the point neurons to the full compartmental models, the parameters of Eqs 24 and 25 are fitted to each cell in the population of models using the dynamic IV-curve method described by Badel et al [141] for threshold properties (S6(A) Fig) and the pattern search global optimisation algorithm for the remaining adaptation properties (S6(B) Fig). The latencies and transfer resistances for dendritic inputs from the lateral and medial perforant paths are also estimated (S6(C) and S6(D) Fig). This produces a joint distribution of parameters across the populations of mature and adult-born neurons (S6(E) Fig), from which sample neurons can be drawn to reflect the statistics and heterogeneity of the full compartmental models [142].

Unless otherwise stated, connectivity is as above. Groups of 10 granule cells, initially 8 mature and 2 adult-born receive common inputs. Inputs are generated with grid-cell like firing patterns (see below). Basket cells are now much rarer, with 100 basket cells in the 50, 000 granule cell model and 200 in the 100, 000 granule cell model. This better reflects their proportions in the real dentate gyrus [5]. Synapses to basket cells are therefore weakened by a factor of ten, and each basket cell typically makes 500 contacts onto granule cells.