An Investigation on Spiking Neural Networks Based on the Izhikevich Neuronal Model: Spiking Processing and Hardware Approach

Abstract

The main required organ of the biological system is the Central Nervous System (CNS), which can influence the other basic organs in the human body. The basic elements of this important organ are neurons, synapses, and glias (such as astrocytes, which are the highest percentage of glias in the human brain). Investigating, modeling, simulation, and hardware implementation (realization) of different parts of the CNS are important in case of achieving a comprehensive neuronal system that is capable of emulating all aspects of the real nervous system. This paper uses a basic neuron model called the Izhikevich neuronal model to achieve a high copy of the primary nervous block, which is capable of regenerating the behaviors of the human brain. The proposed approach can regenerate all aspects of the Izhikevich neuron in high similarity degree and performances. The new model is based on Look-Up Table (LUT) modeling of the mathematical neuromorphic systems, which can be realized in a high degree of correlation with the original model. The proposed procedure is considered in three cases: 100 points LUT modeling, 1000 points LUT modeling, and 10,000 points LUT modeling. Indeed, by removing the high-cost functions in the original model, the presented model can be implemented in a low-error, high-speed, and low-area resources state in comparison with the original system. To test and validate the proposed final hardware, a digital FPGA board (Xilinx Virtex-II FPGA board) is used. Digital hardware synthesis illustrates that our presented approach can follow the Izhikevich neuron in a high-speed state (more than the original model), increase efficiency, and also reduce overhead costs. Implementation results show the overall saving of $84.30\%$ in FPGA and also the higher frequency of the proposed model of about 264 MHz, which is significantly higher than the original model, 28 MHz.

1. Introduction

Neuromorphic engineering is a basic science field that uses different aspects of science such as biological systems, mathematical approaches, physical sciences, computer engineering, electrical, electronic, and digital engineering. On the other hand, in this field of research, different aspects of sciences are connected together for achieving a comprehensive practical system [1,2,3,4,5,6,7,8,9,10].

The main required organ of the biological system is the Central Nervous System (CNS), which can influence the other basic organs in the human body. This complex system includes some basic blocks such as neurons, synapses, and glial cells. In this way, the role of neurons as primary blocks is very important. The neuron is the basic part of this system that can connect to other neurons by a connection called a synapse. Indeed, neurons are responsible for receiving, processing, and sending information [11,12,13,14,15,16,17,18,19].

The behaviors of neurons and other neuronal cells and connections can be modeled and explained by some first-order and simple differential equations [20,21,22,23,24,25,26,27,28]. Among these neural modeling, the Integrate and Fire modeling is simple [20]. On the other hand, the Hodgkin–Huxley (HH) neuron model [28] is a very complex and complete neuron model. The other neuron models are in the range of these two neuron models in case of biological complexity and mathematical equations and also high-accuracy modeling. For example, the Izhikevich neuron model is a simple and highly accurate model in case of reproducing all firing patterns of the brain spiking activity [3,13,19]. Exponential-based models such as AdEx neuron [8] produce the firing activity by use of an exponential function in the mathematical equations. Indeed, different neuron models can be selected to evaluate the CNS. In this approach, two basic parameters must be considered: first is the degree of model complexity in terms of hardware realization, and second is the required resources for implementing different parts of the neuron model. In other words, it may be a very accurate mathematical model, but it requires a lot of hardware resources. Consequently, these two basic issues must be taken into account simultaneously. Among different neuronal models with different states of complexity and accuracy, the Izhikevich neuron model can be a suitable choice. This two-coupled neuron model is capable of producing all patterns of brain signals such as tonic spiking, tonic bursting, phasic spiking, phasic bursting, etc. Indeed, using this model, different dynamics of the human brain can be regenerated.

To realize and implement the neuronal models in a hardware state, there are different choices. Indeed, to achieve a hardware form of the mathematical neuronal models, we have two cases: analog implementation and digital realization. For realizing in the analog case, CMOS elements are applied to achieve an analog architecture to follow mathematical modeling of the neuron. This solution is high speed, but it may be suffer from long development requirements [18]. On the other hand, in the digital realization of neuronal models, a high amount of silicon may be required as well as high power consumption, but this solution can be very efficient in comparison to other ways. Some capabilities of digital implementation include the high degree of flexibility as well as reduced time and power requirements. In this approach, programmable boards such as FPGAs can be very fast and flexible [3,4,5,6,7,8,9,10,11,12,13,14].

A digital realization (implementation) of the Izhikevich neuron model is evaluated in this paper based on LUT modeling. Generally, in digital realization of the neuronal model, it is optimized to approximate the nonlinear terms of the mathematical functions to reach a simple, efficient, and fast design. In other words, the nonlinear terms in the neuronal equations cause a reduction in system performances (in case of frequency and required resources). Thus, the basic challenge for implementing the digital system is converting the nonlinear terms of the models to achieve a comprehensive and high-speed nervous system. In this way, nonlinear terms include polynomial functions with the power of 2 and more, multipliers, divisors, exponential and logarithmic functions, trigonometric functions, and any other nonlinear functions. In our selected model (Izhikevich neuron model), the basic issue is the existence of a quadratic function in the voltage equation, which makes its a nonlinear model. This nonlinear model can be implemented in hardware platforms (such as FPGAs), but to achieve a high-speed system, the nonlinear term can be approximated. In this approach, LUT modeling of quadratic terms is considered. This approach is based on the issue that the nonlinear term in the Izhikevich neuron is not approximated by linear terms, but it can be replaced by LUT-based values. On the other hand, since the LUT-based system is applied, the final error will be reduced, significantly, and also, the performance of the system will be increased in case of high-speed and low-cost final neuronal architecture. As a result, by modifying the original Izhikevich neuron model, we have a new high-speed, low-error, and low-area hardware that can be implemented on FPGA boards as a compact digital design. In large-scale realization, this low-cost and low-area system can be considered for achieving real neural networks. These networks have the same biological behaviors as real neural networks.

Implementation of the Izhikevich neuron model is reported in different papers [3,13,19]. In these papers, the nonlinear term of the Izhikevich neuron model is replaced by some linear and power of 2 functions for better digital realization of the model. On the other hand, when a model is approximated by some simple terms, the error level in implementation may be increased. In addition, the required resources in FPGA platforms are increased significantly, and this affects the final implemented neurons that are tested on an FPGA board. Indeed, a main factor in hardware realization of neuron models on FPGA platforms is increasing the final realized neuron models on one FPGA system. In our procedure, LUT modeling of the Izhikevich neuron is applied. In this methodology, the nonlinear term of the Izhikevich model (quadratic term) is not approximated, but it can be replaced by a Look-Up Table (a simple memory) to have the same function (full similarity). Using this strategy, the error level can be reduced significantly. Consequently, we have an accurate model (similar to the original model) that can reproduce all firing patterns in a high degree of similarity. This new model is implemented on a Xilinx Virtex-II FPGA board (XC2VP30 model) without any nonlinear terms, resulting in a biological neural network with high efficiency and frequency (as well as low-cost implementation).

In this approach, piecewise linear, hyperbolic-based, and dynamic-based and other methods are available, but in all of these methods, the original functions are replaced by new terms with similar behaviors of the original one. This may cause some error levels, since the original terms have been removed and modified. In neural networks, the error level is important, and using the previous methods causes the increasing errors in calculations. Consequently, LUT-based model can be applied to achieve a high-similarity level between the original and proposed models. In other words, using LUT-based functions, we have the same original model with new aspects that cause a high level of reliability. Indeed, this proposed model mimics the original model with a high degree of accuracy and low-error state in comparison to other approximation methods.

This paper is presented as follows. In Section 2, the original Izhikevich neuron will be explained. In Section 3, the proposed strategy is evaluated. Section 4 presents the error analysis. The hardware design procedure is performed in Section 5 in detail. Final results are presented in Section 6. The network view and conclusion are given in Section 7 and Section 8, respectively.

2. Izhikevich Neuronal Model

In this approach, an Izhikevich neuronal model has been considered [3,13,19]. This model uses two coupled first-order differential equations to extract the neuron voltages. Some fixed parameters such as $a,b,c,$ and d are applied to extract all 20 firing patterns of the human brain. The following equations are responsible for producing these output signals:

$$\frac{dv}{dt}=0.04v^2+5v+140-u+I$$

(1)

$$\frac{du}{dt}=a(bv-u)$$

(2)

with conditional equations:

$$ifv>v_{th}then\left\{\begin{array}{c}v⟵c\hfill \\ u⟵u+d\hfill \end{array}.\right.$$

(3)

In these equations, v is the voltage variable, and u is the recovery state. In addition, $I$ is a stimulus current for triggering the neuronal cell. As mentioned, the Izhikevich neuron is adapted to regenerate the 20 spiking activity of the human brain. This system can achieve this phenomenon by varying different parameters in its mathematical modeling ($a,b,c,d,$ and I). Some different spiking patterns that can be extracted by the Izhikevich neuron model are depicted in Figure 1. In addition, based on

Table 1. Suitable parameters for spiking signals in the Izhikevich neuron model.

Neuron Type	a	b	c	d
Tonic patterns of spiking	0.02	0.20	−65	6
Phasic patterns of spiking	0.02	0.25	−65	6
Tonic bursting	0.02	0.20	−50	2
Phasic bursting	0.02	0.25	−55	0.05
Mixed mode	0.02	0.20	−55	4
SFA	0.01	0.20	−65	8
Class 1	0.02	−0.10	−55	6
Class 2	0.20	0.26	−65	0
Spike latency	0.02	0.20	−65	6
Sub. Osc.	0.05	0.26	−60	0
Resonator	0.10	0.26	−60	−1
Integrator	0.02	−0.10	−55	6
Rebound spike	0.03	0.25	−60	4
Rebound burst	0.03	0.25	−52	0
Threshold variability	0.03	0.25	−60	4
Bistability	1	1.50	−60	0
DAP	1	0.20	−60	−21
Accomodation	0.02	1	−55	4
IIS	−0.02	−1	−60	8
IIB	−0.02	−1	−45	0

, different parameters of the Izhikevich model for generating all 20 spiking patterns are presented.

3. Proposed Strategy

The basic nonlinear functions in the mathematical equation of the neuronal systems cause the reduction in the frequency (speed-up) and large number of digital implemented neurons (in case of using a high amount of FPGA resources). In other words, for achieving a simple and low-error system with a high degree of similarity with the original neuron model, it is expected that the nonlinear terms of the model are generated using a compact memory (LUT modeling) without any mathematical approximation. The basic challenge of implementation of the Izhikevich neuron model (in high-speed level and low-cost realization) is converting the quadratic term ($v^2$) of this model. This approach is based on LUT modeling (compact memory) in three cases: 100 points LUT modeling, 1000 points LUT modeling, and 10,000 points LUT one. Indeed, the quadratic term of the Izhikevich model is divided into these selected points (100, 1000, and 10,000 points) as a digital sampling of the original function. Thus, the voltage equation of the original model can be reformulated as follows:

$$\frac{dv}{dt}=LUT\left(v\right)+Lin\left(v\right)-u+I_{trigger}$$

(4)

where

$$\left\{\begin{array}{c}LUT\left(v\right)=0.04v^2\left[k\right];1<k<k_{max}\hfill \\ Lin\left(v\right)=4v+v+140\hfill \end{array}.\right.$$

(5)

In this formulation, $0.04v^2\left[k\right]$ is a unique value based on different k points ($1<k<k_{max}$ in which $k_{max}$ can be 100, 1000, or 10,000 depending on the different accuracy). In addition, $Lin\left(v\right)$ is a linear simple term that can be implemented using only digital shifters and adders (more explanations are presented in next sections). As depicted in Figure 2, three sampling cases for replacing the nonlinear term ($0.04v^2$) with the LUT state are presented. As can be seen in this figure, as the $k_{max}$ is increased, the sampling points in the LUT(v) function are increased and the accuracy of the proposed modeling goes higher and higher. On the other hand, using this method, each point is selected, calculated, and assessed. After computing each point value, its calculation amount is applied to the voltage variable of the Izhikevich neuron model for reproducing different spiking patterns.

4. Error Analysis

The proposed model that is replaced with the original one must trace the all behaviors of the firing patterns. In addition, in a memory case of neural networks, the spiking activity on the exact timing process is very important, and any delay in this issue causes many problems in synaptic coupling between two coupled neurons.

There are several error methods [1,2,3,4,5,6,7,8,9,10,11,12,13,14] to validate the proposed model capability (in terms of spiking activity tracing):

• Root-Mean-Square Error (RMSE): This error criteria is applied for calculating the differences between the proposed and original output signals in terms of quadratic and square state.
• Mean Absolute Error (MAE): This error criteria is applied for computing the absolute differences between the original and the proposed output signals in terms of high similarity.
• Correlation is a relationship between the proposed and original neuron models in terms of the following similarities. Indeed, when a correlation factor between two signals is high, it is demonstrated that these two signals are in the same behavior states.

These formulations are presented in

Table 2. Different error methods.

$\mathbf{RMSE}({\mathbf{v}}_{\mathbf{prop}},{\mathbf{v}}_{\mathbf{orig}})=\sqrt{\frac{{\sum }_{\mathit{i}=1}^{\mathit{n}}{\left({\mathit{v}}_{\mathit{p}\mathit{r}\mathit{o}\mathit{p}}-{\mathit{v}}_{\mathit{o}\mathit{r}\mathit{i}\mathit{g}}\right)}^2}{\mathit{n}}}$
$\mathbf{MAE}=\frac{1}{n}{\sum }_{i=1}^n|v_{prop}-v_{orig}|$
$\mathbf{Corr}({\mathbf{v}}_{\mathbf{prop}},{\mathbf{v}}_{\mathbf{orig}})=\frac{COV(v_{orig},v_{prop})}{{\sigma }_{orig}{\sigma }_{prop}}$

.

Error calculation by these methods must be performed. For these computations, there is a sampling rate, which is calculated by $\mathbf{dt}$. This parameter is important for calculating different errors. In this state, this parameter is considered as $\mathbf{dt}=\mathbf{5}\mathbf{ms}$. To validate the performance of the proposed approach, some firing activities of the original and proposed models have been presented. In this way, Figure 3 depicts this comparison in high accuracy. As illustrated in this figure, the proposed model can follow the original system with minimum error. In addition, for all 20 firing patterns of the Izhikevich neuron model, the error criteria are calculated in

Table 3. RMSE, MAE, and Corr computations for different spiking patterns.

Firing Type	RMSE	MAE	Correlation %
Tonic spiking	0.8	0.20	95
Phasic spiking	0.7	0.25	91
Tonic bursting	0.6	0.20	90
Phasic bursting	1.1	0.35	93
Mixed mode	1.02	0.26	98
SFA	1.01	1.20	94
Class 1	1.02	1.10	89
Class 2	0.20	1.26	88
Spike latency	0.2	1.20	89
Sub. Osc.	0.5	1.26	91
Resonator	0.10	2.26	90
Integrator	0.2	1.10	85
Rebound spike	0.33	0.1	96
Rebound burst	0.57	0.28	82
Threshold variability	0.13	0.43	91
Bistability	1	1.50	91
DAP	1	1.20	93
Accomodation	0.32	1	94
IIS	0.12	1	87
IIB	1.8	0.9	88

.

5. Hardware Design Procedure

This section presents a procedure for implementing the proposed neuronal model on the FPGA platform. FPGAs are in the category of the ICs (Integrated Circuits) that are used in flexible form and can be configured as a programmable device. This flexible property that causes the FPGA is a suitable choice for implementing the neuronal models. Indeed, in this structure, a Hardware Description Language (HDL) is considered to achieve a flexible and programmable final circuit [1,2,3,4,5,6,7,8,9,28]. The internal structure of an FPGA board includes an array of LBs (Logic Blocks), which are configurable (configurable logic blocks). In fact, the basic logical elements such as AND and XOR are used in this structure. In this way, to realize the proposed Izhikevich neuron model, a Virtex-II FPGA board (XC2VP30 model) has been used.

For implementing the neuronal model, several steps are evaluated:

• Equation Discretizing: The first step for implementing the proposed neuronal model in a digital state is equation discretizing. On the other hand, it is a process in which all proposed equations and differential models are converted to digitized form to prepare for applicationr in digital platforms.
• Bit-Width Definition: This process is very important in case of avoiding any overflow in digital data transferring.
• Hardware Architecture: This is a main step in digital hardware realization. Indeed, different parts of the final digital system have been designed and implemented in this state.

5.1. Discretizing the Equations

Equations discretizing is a first step in digital implementation. In fact, by discretizing the final variables of the model, it can be prepared for designing the digital circuits. There are different ways to discretize the equations. In this approach, the Euler method [1,2,3,4,5,6,7,8,9,10,11,12] is used for solving the differential equations of two basic variable, V and U. This method is based on digitizing the differential equations of the model based on one step size. The basic equations of the proposed model (V and U) are given by

$$V[i+1]=dt\ast [LUT\left(V\left[i\right]\right)+Lin\left(V\left[i\right]\right)-U\left[i\right]+I_{trigger}]+V\left[i\right]$$

(6)

$$U[i+1]=dt\ast \left[a\right(bV\left[i\right]-U\left[i\right]\left)\right]+U\left[i\right]$$

(7)

where

$$\left\{\begin{array}{c}LUT\left(V\left[i\right]\right)=0.04V{\left[i\right]}^2\left(k\right);1<k<k_{max}\hfill \\ Lin\left(V\right[i\left]\right)=4V\left[i\right]+V\left[i\right]+140\hfill \end{array}.\right.$$

(8)

5.2. Bit-Width Definition

This issue is very important in digital hardware design. In our proposed neuron model, a fixed-point approach is evaluated. To attain a reliable system without any data loss, two basic points must be taken into account. First, there is the range of data variation for any variable and parameter in the system. For example, for our design, the variation range of variables and parameters is between $(-70)$ and $(+30)$. This variation range needs almost 8 bits for integer computations and 3 bits for the fractional computations. On the other hand, the second issue is the maximum digital shifts of the variables to right or left. Based on our digital design, the maximum digital shift to the right is 7 bits, and the maximum digital shift to the left is 3 bits. Consequently, these two numbers of shifts are added to the basic bit-width of the system, creating the final bit-width. By this calculation, the final bit-width of the system is 22 bits. This bit-width is divided into 11 bits for the integer part, 10 bits for the fraction part, and 1 bit for the number sign.

5.3. Hardware Architecture

This section is the basic part of the final digital implementation. Indeed, the digital hardware details are explained and evaluated in this approach. In a general case, the architecture of the digital system (specially for the neuronal system) is composed of some sections as follows:

• Input Data Part (IDP): This part stores required data about input parameters.
• Neuron Voltage Calculation (NVC): This section provides the output data for neurons voltage.
• Control Signal Generator (CSG): This part controls the clock and reset pulses and also reset conditions for neuron voltages.
• Look-Up Table Data Computation (LUTDC): This unit computes the required data for producing nonlinear functions.
• Linear Part Generator (LPG): This part generates the linear term of the Izhikevich neuron model.
• Output Data Part (ODP): This unit stores the output production data.
• Output Display Unit (ODU): This part displays the final signals.

All the above parts are explained in details in the next subsections. In addition, the overall architecture of the final system is shown in Figure 4. Different parts of the presented hardware are in direct communication with them, and the data computations in each part can influence on the other sections.

Different sections of the proposed architecture are explained in details in the next subsections.

5.4. Input Data Part (IDP)

This unit is responsible for storing the initial values of the basic variables and parameters of the proposed model such as V, U, $V_{th}=30mv$, and $\mathrm{Par}=140$. Moreover, the parameters of the Izhikevich model for generating all 20 spiking patterns can be stored in 100 SRAMs. The basic parameters for reproducing these patterns are a, b, c, d, and I. An enable selector is also required for selecting the final state for transferring its data to the NVC. This procedure is depicted in Figure 5.

5.5. Neuron Voltage Calculation (NVC)

This part concludes the output neuron voltage. As can be seen in Figure 6, based on the signals that can be generated by LUTDC and LPG parts, the required signals (V and U) have been generated. Based on the proposed equations of the Izhikevich neuron model, the scheduling diagrams of this new model are presented. In this architecture, the pipeline system has been realized. Thus, the frequency of the designed system will be increased, significantly.

5.6. Control Signal Generator (CSG)

This important part generates the CLOCK and Reset pulses for all parts of the system based on the FPGA platform. On the other hand, it is responsible for controlling the voltage generation in the NVC unit. In other words, since the neuron voltage is varied between $-70$ and $+30$ mv, when the voltage reaches a high range, it must be reset to a fixed value. Thus, this part controls the spiking patterns, which are generated by the NVC unit. A graphical view of this presentation is illustrated in Figure 7.

5.7. Look-Up Table Data Computation (LUTDC)

This part is a main section for implementing our modified model. Indeed, the nonlinear term of the original Izhikevich neuron model ($0.04V^2$) is replaced by the LUTDC module. As mentioned, three different methods (based on different accuracy) are considered. Based on the selection of $K_{max}$, this nonlinear term will be replaced by some digitized values. As can be seen in Figure 8, this part contains memory. Different accuracy is selected, and then, the required value will be extracted.

5.8. Linear Part Generator (LPG)

This simple part of the original model is realized only by digital shifts and adds. Indeed, without any use of multipliers, this linear part will be implemented. The proposed hardware is depicted in Figure 9.

5.9. Output Data Part (ODP)

After generation of the neuron voltage from the NVC unit, the final output signals are transferred to this part. Indeed, ODP stores the output data. This architecture is depicted in Figure 10.

5.10. Output Display Unit (ODU)

Finally, after storing required data in the ODP unit, final signals can be displayed in ODU. On the other hand, digital data from ODP are applied to the ODU (digital 8-bit DAC), and then, these signals will be shown on a digital oscilloscope. Figure 11 shows this unit.

6. Final Results

The digital final model is implemented on the Xilinx Virtex-II (XC2VP30) FPGA board. In this way, Figure 12 shows the digital display of the neuron voltages for some patterns of the Izhikevich model.

There are several papers that implement this Izhikevich neuron model. For example, Soleimani et al. [13] proposes a set of piecewise linear models for realizing the Izhikevich neuron in high performances. In fact, they have focused on converting the nonlinear part of the original model with some linear functions. This approach may causes errors in the proposed model because of the differences for two models (proposed and original) by some ranges. The proposed model is implemented on a FPGA Virtex-II (XC2VP30 model). On the other hand, Junran Pu et al. [19] presents a new approach for implementing the Izhikevich neuron with hyperbolic functions. In their approach, a set of hyperbolic functions are applied for removing the nonlinearity of the quadratic terms of the original model. The hyperbolic functions can be converted to power-2 based functions, and this causes multiplier-less digital realization. In other words, these power-2 based functions convert to digital shifts and adds. This hyperbolic function is a good approach in case of reducing the final error computations. However, it may increase the final hardware resources. In addition, in [19], a Xilinx Zynq-7000 SoC ZC702 FPGA has been used. It is emphasized that in [19], the number of FPGA resources are higher than our FPGA board (Spartan3), and if we used the Zynq-7000 board, the frequency and costs will be organized in a better state. In our approach, since the nonlinear term of the Izhkevich neuron model is replaced by a memory (LUT-based), the error calculations will be reduced, significantly. On the other hand, since the proposed model is composed of some digitized values (as LUT), the final required resources will be reduced. To validate the proposed model (50 implemented neurons),

Table 4. Low-level device utilization of XILINX Virtex-II Pro and Zynq-7000; the used percentage of basic elements for implementation of the neuron model.

Resources	Our Model	Izh. Model	Soleimani [12]	Junran Pu [19] (Zynq-7000)
Number of Slices	$302(2.2\%)$	$857(6\%)$	$495(3\%)$	107
Number of Slice Flip Flops	$224(0.8\%)$	$551(2\%)$	$493(1.8\%)$	341
Number of 4-input LUTs	$468(1.7\%)$	$1268(4.6\%)$	$617(2.2\%)$	343
Number of bonded IOBs	$27(5\%)$	$34(6\%)$	$34(6\%)$	NA
Number of GCLKs	$1(6\%)$	$1(6\%)$	$1(6\%)$	1
MULT18*18s	$0(0\%)$	$4(3\%)$	$0(0\%)$	$0(0\%)$
Overall Saving	$84.30\%$	$72\%$	$81\%$	NA
Max speed (Total available: 400 MHz)	264 MHz (66%)	28 MHz (7%)	242 MHz (60%)	224 MHz

is presented. As can be seen in this table, the frequency of the proposed system is increased, significantly. In addition, the overall saving in FPGA resources is in a good state in comparison with the original model and other comparable models.

In this realization, since the proposed model is a multiplierless design, the final power consumption will be reduced significantly (based on the removing of all nonlinear terms that are high-cost terms) compared with the original neuronal model.

One of the basic parameters in the realization of neural networks is the large-scale implementation. In this approach, the overall saving in the FPGA resources is an essential issue. In this paper, the overall saving in FPGA resources is 84.30%, which is higher than the original and other paper models. On the other hand, using an FPGA board, the maximum number of implemented proposed neurons is higher than in other models. Consequently, the proposed model is in the better state in case of large number of implemented neurons compared to other methods.

The implementation of neuronal networks is an attractive research area. In this approach, neural networks with a large number of neurons have been connected to achieve a real system. This issue can be used for investigating the neuronal diseases. On the other hand, using this proposed high-speed and low-cost system, we have a compact design that can be applied in investigating the different aspects of real neural networks.

7. Network View

The populations of randomly connected neurons can be investigated in terms of different spiking behaviors in the biological system [1,2,3,4,5,6,13,14,15,24]. In this methodology, 1000 randomly connected neurons are considered, and for the proposed model, they are tested. After simulation of this network, as can be seen in Figure 13, the original and the proposed model behaviors are the same. To investigate the network dynamics of the neurons with the proposed LUT-based model and compare it with the original neuron model, a network of randomly connected 1000 neurons is simulated. Motivated by the anatomy of a mammalian cortex, we choose the ratio of excitatory to inhibitory neurons to be 4 to 1 and make the inhibitory synaptic connections stronger (800 excitatory with 200 inhibitory neurons).

For evaluating the error calculation in the network state, a different error method can be applied called Mean Relative Error (MRE). The error criteria are based on the differences between the proposed and the original firing patterns in the network area. The error formulation is given as follows:

$${\mathbf{MRE}}_{\left(\mathbf{proposed}\right)}\%=\frac{{\sum }_{\mathbf{i}=\mathbf{1}}^{\mathbf{N}}\frac{|\Delta {\mathbf{t}}_{\left({\mathbf{proposed}}_{\mathbf{i}}\right)}|}{|{\mathbf{t}}_{\mathbf{si}}|}}{\mathbf{N}}\ast \mathbf{100}.$$

(9)

In this equation, differences between the ith pattern is captured by parameter $\Delta$t. In addition, the parameter N shows the sampling number. This error computation can be calculated for all 20 firing patterns of the proposed model, and

Table 5. Relative error calculations for all firing behaviors of the proposed neuron model.

Neuron Type	MRE %
Tonic patterns of spiking	0.85
Phasic patterns of spiking	0.32
Tonic patterns of bursting	0.24
Phasic patterns of bursting	0.23
Patterns of mixed mode	0.43
Patterns of spike frequency adaptation	0.87
Class I patterns	0.36
Class II patterns	0.27
Spike latency patterns	0.75
Subthreshold oscillation patterns	0.23
Resonator patterns	0.85
Integrator patterns	0.47
Rebound spike patterns	0.28
Rebound burst patterns	0.79
Threshold variability patterns	0.81
Bistability patterns	1.29
DAP patterns	1.85
Accomodation patterns	0.28
Patterns of inhibition-induced spiking	0.22
Patterns of inhibition-induced bursting	1.43

presents this error for all states.

8. Conclusions

In this paper, an LUT-based version of the Izhikevich neuron model is presented. The proposed architecture can reproduce all 20 signals of the Izhikevich neuron with good similarity and efficiency. Indeed, the proposed model uses a memory for calculating the nonlinear term of the original neuron model. This approach reduces errors in the calculation and increases the performance of the system in case of decreasing the required resources on the FPGA platform. Since the nonlinear parts of the Izhikevich neural modeling have been removed, we have a multiplierless digital implementation. The proposed designed system is in the high-frequency state, and there is also an observable cost reduction in FPGA resources compared with the similar implemented neuron models. To test and validate the proposed final hardware, a digital FPGA board (Xilinx Virtex-II FPGA board) is used. Digital hardware synthesis illustrates that our presented approach can follow the Izhikevich neuron in a high-speed state (more than the original model), has better efficiency, and also reduces overhead costs. Implementation results show the overall saving of $84.30\%$ in FPGA and also the higher frequency of the proposed model, about 264 MHz, which is significantly higher than the original model, 28 MHz.