Fig. 1. Performance of different types of neural microcircuit models with linear readouts for classification of spike patterns. (a) In the top row are two examples of the 80 spike patterns that were used as templates for noisy variations (each consisting of 4 Poisson spike trains at 20 Hz over 200 ms), and in the bottom row are examples of noisy variations (Gaussian jitter with SD 10 ms) of these spike patterns which were used as circuit inputs. (b) Fraction of examples (for 500 test examples) for which the output of a linear readout (trained by linear regression with 2000 training examples) agreed with the target classification. Results are shown for 90 different types of neural microcircuits C with λ varying on the x-axis and on the y-axis (20 randomly drawn circuits and 20 target classification functions randomly drawn from the set of 2⁸⁰ possible classification functions were tested for each of the 90 different circuit types, and resulting correctness-rates were averaged). Circles mark three specific choices of λ, -pairs for comparison with other figures, see Fig. 3. The standard deviation of the result is shown in the inset on the upper right. (c) Average firing rate of the neurons in the same circuits and with the same inputs as considered in (b). One sees that the computational performance becomes best at relatively low average firing rates around 4 Hz (see contour line), and that the region of best performance cannot be characterized in terms of the resulting average firing rate.

Fig. 2. UP- and DOWN-states in neural microcircuit models. Membrane potential (for a firing threshold of 15 mV) of two randomly selected neurons from circuits in two parameter regimes labeled as UP- and DOWN-states, as well as spike rasters for the same two parameter regimes (with the actual circuit inputs shown between the two rows).

Fig. 3. Analysis of changes in the dynamics resulting from small input differences for different types of neural microcircuit models as specified in Section 2. Each circuit C was tested for two arrays u and v of 4 input spike trains at 20 Hz over 10 s that differed only in the timing of a single spike at time . (a) A spike at time was delayed by 0.5 ms. Temporal evolution of Euclidean differences between resulting circuit states and with 3 different values of λ, according to the 3 points marked in panel (c). For each parameter pair, the average state difference (thick line) and standard deviation (thin line) of 40 randomly drawn circuits is plotted. (b) Lyapunov exponents μ along a straight line between the points marked in panel (c) with different delays of the delayed spike. The delay is denoted on the right of each line. The exponents were determined for the average state difference of 40 randomly drawn circuits. (c) Lyapunov exponents μ for 90 different types of neural microcircuits C with λ varying on the x-axis and on the y-axis (the exponents were determined for the average state difference of 20 randomly drawn circuits for each parameter pair). A spike in u at time was delayed by 0.5 ms. The contour lines indicate where μ crosses the values −1, 0, and 1. (d) Computational performance of a linear readout with these circuits (same as Fig. 1(b)), shown for comparison with panel (c).

Fig. 4. VC-dimension as predictor for the generalization capability of neural microcircuit models. (a) Prediction of the generalization capability for 90 different types of neural microcircuits (as in Fig. 1(b)): estimated VC-dimension (of the hypothesis class for a set of inputs consisting of 500 jittered versions of 4 spike patterns), each of the 90 data points for 90 different circuit types was computed as the average over 20 circuits; for each circuit, the average over 5 different sets of spike patterns was used. The standard deviation is shown in the inset on the upper right. See Section 6 for details. (b) Actual generalization capability ((error on test set) − (error on training set)) of the same neural microcircuit models for a particular learning task, quantified by the difference of test error and training error (error defined as the fraction of examples that are misclassified) in the spike pattern classification task discussed in Section 2. The standard deviation is shown in the inset on the upper right.

Fig. 5. Values of the proposed measures for computations on spike patterns. (a) Kernel-quality for spike patterns of 90 different circuit types (average over 20 circuits, mean SD=13). ⁷ (b) Generalization capability for spike patterns: estimated VC-dimension of (for a set of inputs u consisting of 500 jittered versions of 4 spike patterns), for 90 different circuit types (same as Fig. 4(a)). (c) Difference of both measures (the standard deviation is shown in the inset on the upper right). This should be compared with actual computational performance plotted in Fig. 1(b).

Fig. 6. Analysis of the sets of neurons which get activated. (a) For a given input pattern, only a small fraction of the 540 circuit neurons are activated (i.e., emit at least one spike). Shown is the mean number of neurons which are activated by an input pattern. For each parameter setting, the mean over 20 circuits and 125 patterns per circuit is shown. The input patterns were drawn from the same distribution as the templates for the task in Fig. 1. (b) For different input patterns, sets of activated neurons are diverse. Shown is the mean Hamming distance between two of the 125 activation vectors arising from a set of circuit inputs consisting of 125 jittered versions of a spike template. For each parameter setting, the mean over 20 circuits is shown. (c) Difference between the mean Hamming distance for an input set consisting of 125 randomly drawn spike templates and the mean Hamming distance for an input set consisting of 125 jittered versions of a spike template. This difference predicts computational performance (see Fig. 1(b)) quite well. For each parameter setting, the mean over 20 circuits is shown. The standard deviation is shown in the inset on the upper right.

Fig. 7. Performance and values of the proposed measures for different types of neural microcircuit models with linear readouts for classification of spike patterns on the basis of circuit states with limited precision. In each case the actual analog value of each component of the circuit state was rounded to one of 11 possible values (see text). (a) Fraction of examples (for 500 test examples) for which the output of the readout agreed with the target classification (the readout was trained by linear regression with 2000 training examples; see Fig. 1(b) for more details). Note the similarity to Fig. 1(b). The standard deviation of the result is shown in the inset on the upper right. (b) Generalization capability of the same neural microcircuit models for a particular learning task, quantified by the difference of test error and training error (error defined as the fraction of examples that are misclassified). Compare with Fig. 4(b). (c) Kernel-quality for spike patterns of 90 different circuit types. (d) Generalization capability for spike patterns. (e) Difference of both measures (the standard deviation is shown in the inset on the upper right). Compare with Fig. 5(c).

Fig. 8. Analysis of the computational performance of simulated neural microcircuits (together with linear readouts) in different dynamic regimes. (a) Estimates of the kernel-quality for 135 spike patterns (solid line; average over 20 circuits). Estimate of the VC-dimension for a set of inputs consisting of 135 jittered versions of one spike pattern (dotted line; average over 20 circuits). (b) Difference of measures from panel (a) (as in Fig. 5(c); mean ± SD). (c) Evaluation of computational performance (correlation coefficient; for 500 test examples) of a linear readout (trained by linear regression with 1000 training examples). 20 randomly drawn circuits and 20 target classification functions randomly drawn from the set of 2³⁰ possible classification functions were tested for each of the 13 different circuit types, and resulting correlation coefficients were averaged. (d) Estimates of the kernel-quality for input streams u with 13² different combinations of 13 firing rates (solid line; average over 20 circuits). Estimate of the VC-dimension for a set of inputs consisting of 13² different spike trains u that represent one combination of firing rates. (dotted line; average over 20 circuits). Also shown is the average number of neurons that get activated (i.e. fire at all) for a typical input (dash-dotted line; scale on right hand side). (e) Difference of measures from panel (d) (solid line). Average firing rate in the same circuits for the same inputs (dashed line). Note that the firing rates are poorly correlated both with the difference of the two measures, and with the computational performance shown in (f). (f) Evaluation of computational performance (correlation coefficient; for 400 test examples) of a linear readout (trained by linear regression with 2000 training examples). 20 randomly drawn circuits were tested for each of the 13 different circuit types, and resulting correlation coefficients were averaged. Results are shown for classification at three different time points after stimulus onset: 100 ms (solid line), 150 ms (dashed line), and 200 ms (dash-dotted line).This shows that at least for this task performances for these time points were comparable. Their performance peak (as a function of the amount of background noise) was also predicted quite well by our two measures as shown in panel (e).