Ternary Neural Networks Based on on/off Memristors: Set-Up and Training

Abstract

Neuromorphic systems based on hardware neural networks (HNNs) are expected to be an energy and time-efficient computing architecture for solving complex tasks. In this paper, we consider the implementation of deep neural networks (DNNs) using crossbar arrays of memristors. More specifically, we considered the case where such devices can be configured in just two states: the low-resistance state (LRS) and the high-resistance state (HRS). HNNs suffer from several non-idealities that need to be solved when mapping our software-based models. A clear example in memristor-based neural networks is conductance variability, which is inherent to resistive switching devices, so achieving good performance in an HNN largely depends on the development of reliable weight storage or, alternatively, mitigation techniques against weight uncertainty. In this manuscript, we provide guidelines for a system-level designer where we take into account several issues related to the set-up of the HNN, such as what the appropriate conductance value in the LRS is or the adaptive conversion of current outputs at one stage to input voltages for the next stage. A second contribution is the training of the system, which is performed via offline learning, and considering the hardware imperfections, which in this case are conductance fluctuations. Finally, the resulting inference system is tested in two well-known databases from MNIST, showing that is competitive in terms of classification performance against the software-based counterpart. Additional advice and insights on system tuning and expected performance are given throughout the paper.

1. Introduction

Neuromorphic computing, which imitates the principle behind biological synapses with a high degree of parallelism, has recently emerged as a very promising candidate for novel and sustainable computing technologies [1]. Among these technologies, neuromorphic systems based on hardware neural networks (HNNs) implemented with memristive devices have emerged as a promising solution for building energy-efficient computing frameworks for solving most of the tasks carried out in machine learning [2,3,4]. This is because memristors (1) behave as a resistor with memory that is electrically programmable and matches the functionality of the connections in a software neural network and (2) are efficiently integrated thanks to the crossbar array structure (i.e., aggressive size scaling is possible) [1,5,6].

Focusing on specific implementations that use memristors based on the crossbar array structure, it is worth noting first that this is one common approach found in the literature [7]. By using this structure, vector-matrix multiplications, which are a fundamental building block in all types of neural networks, are efficiently implemented by following an analog approach (i.e., by adding current flows). The efficiency of the operation is both in terms of (1) power consumption, because the involved currents are small, and (2) computational time [8], because the whole operation is performed by reading the outputs of the array. Note that a vector-matrix operation in software has a computational time that is $\mathcal{O}(m·n)$ and that individual memristors in the crossbar array play the role of the matrix coefficients or, in terms of neural networks, the weights. The interested reader can find in [9] a specific sound localization application based on memristor arrays. Energy consumption is reduced a factor of 184 with regard to the existing Application-Specific Integrated Circuit (ASIC) design.

Memristor-based networks can be trained by offline (or ex situ) or online (or in situ)-learning methods. In the first case, which is the focus of this manuscript, the weights are calculated on a precursor software-based network and then imported sequentially into the crossbar circuit. In the second case, training is implemented in situ in hardware and only for small neural networks [10], so the weights are adjusted in parallel, which is significantly more demanding [5]. In both cases, a high precision weight import is required to implement complex networks and achieve the expected performance when the network is operating. However, various properties of memristors are known to negatively affect the performance of neuromorphic systems [1]. Specifically, the conductance response of any real nonvolatile memory (NVM) device exhibits non-idealities that can surface in the form of unreliable performance of the network. Those imperfections include non-linearity, stochasticity, varying maxima, asymmetry between increasing and decreasing responses, and unresponsive devices at low or high conductance [11,12,13,14]. For example, most memristive devices exhibit a nonlinear weight update, where the conductance gradually saturates [1]. In addition, related to HNNs from the perspective of high-performance computing, recent trends show a growing interest in hardware that is capable of accelerating both training and inference in neural networks, especially when dealing with deep learning schemes. That is the case, for example, with many Field-Programmable Gate Array (FPGA) implementations [15], which emphasize the idea of quantized neural network designs due to the nature of FPGA devices. In particular, binary [16] and also ternary [17] implementations have been raised as very interesting options. The main motivation of this alternative approach is the reduction of both power consumption and the FPGA specs (required area). Memristor-based neural networks can also benefit from the power and area. However, the operational principles of memristors are completely different to those found in FPGAs, and at the end of the day, all HNN solutions require solving very specific challenges, as far as a straightforward conversion from the ideal (or software-based) model does not exist. As commented above, one of the challenges in memristor-based neural networks, which work from an analog perspective, is the development of reliable weight implementation due to the variability that is common to all nano-electronic devices but is significantly important in memristors [18].

In that direction, the authors of [4] stated that many issues still need to be resolved at the material, device, and system levels to simultaneously achieve high accuracy, low variability, high speed, energy efficiency, a small area, low cost, and good reliability. Thus, the first step is to obtain memristor-based networks that are competitive in comparison to software-based networks. In order to achieve that, we need to cope with the hardware. This can be accomplished at the hardware level with more advanced mitigation techniques or at the analgorithmic level by taking into account the non-idealities. In that sense, the authors of [10] presented a mask technique to capture the sneak path problem, stating that any kind of training incorporating the knowledge of the crossbar array behavior will likely improve the accuracy of memristor-based networks significantly. This idea was explored by several recent works following different strategies. In [19], for instance, a tailored training method was proposed to address the voltage drop due to the interconnected wire resistance. Basically, the voltage drop is estimated to recompute the weights at the forward propagation stage during the training procedure. In [20], the authors considered the mapping of neural network weights by analyzing the parasitic resistance effects at the different areas of the crossbar array. By identifying those hardware cells providing higher accuracy as “safe zones”, adaptive weight allocation was performed to properly map the weights to the hardware. In [21], the authors mathematically modeled the sensitivity of the output of the neural network with respect to hardware impairments. Then, the cost function of the training algorithm was adapted to consider this sensitivity as an additional term (i.e., the weights were calculated to minimize the impact of hardware impairments as well).

The aim of our work is also to consider hardware impairments during the design and training of the memristor-based neural network. To do so, we depart from software models that emulate the behavior of the memristor-based neural network. More specifically, this work is an extension of the work in [22], and we consider building ternary networks using crossbar arrays. The goal is to achieve performances close to the software models, even when we consider a simple configuration of the memristors operating like ON/OFF switches. It is worth noting that we adopt ternary weights because they have stronger expressive abilities than their binary counterpart [17]. As shown in Section 2, a ternary option does not modify the proposed crossbar array architecture, and the hardware remains the same (i.e., two conductance levels at the memristor weights).

The main contributions of this work are as follows:

• The behavior of a ternary memristor-based HNN adopting crossbar arrays is emulated;
• Practical configuration strategies to tune the crossbar array structure from a system-level designer point of view are proposed;
• An offline (ex situ) training mechanism is derived to optimize the neural network’s weights by minimizing the impact of conductance imperfections in the memristors’ hardware.

In what follows, Section 2 defines the problem under study, including the crossbar array architecture that we are considering to emulate ternary networks. Section 3 the encompasses configuration issues as well as the algorithm considered to fix the memristors to either the ON or the OFF status. Finally, Section 4 provides the experimental results, and Section 5 concludes the paper.

2. Scenario Description and Assumptions

Let us consider a generic feedforward neural network (FFNN) that is dedicated to a classification task, as depicted in Figure 1. The network has inputs $\mathit{x}={[1,x_1,\dots ,x_N]}^T$, where ${(·)}^T$ stands for the matrix transpose and the first input is manually set to 1 in order to accommodate the bias term. The FFNN operates as described next. First, the inputs are linearly combined by means of a matrix multiplication with ${\mathit{W}}^1$, thus generating the values ${\mathit{z}}^1=[z_1^1,\dots ,z_{N_1}^1]$, i.e., ${\mathit{z}}^1={\mathit{W}}^1\mathit{x}$ (see Figure 1). Superindex 1 here stands for the first layer of the network. The values in ${\mathit{z}}^1$ go through a nonlinear function f (typically the sigmoid, the hyperbolic tangent or the rectified linear unit) to generate the activations at the first layer (i.e., ${\mathit{a}}^1=[a_1^1,\dots ,a_{N_1}^1]$ with $a_i^1=f\left(z_i^1\right)$). This process is repeated at the subsequent layers of the FFNN, such as the output at the second layer being computed from ${\mathit{a}}^1$ by first computing ${\mathit{z}}^2={\mathit{W}}^2{\mathit{a}}^1$ and then transforming the values in ${\mathit{z}}^2$ by using the non-linear function f again. Finally, at the last layer, also called the output layer, f is replaced by the softmax function. In this case, the output is normalized (i.e., ${\sum }_{i=1}^{N_J}{a}_i^J=1$), and the value of $a_i^J$ indicates our confidence level in that $\mathit{x}$ corresponds to the ith class. Therefore, the network takes the output with largest value as the resulting classification.

All the operations described above are computed in floating-point arithmetic. We will refer to it as the software implementation. In this work, we employ the crossbar array to compute the vector-matrix multiplications at the neural network layers (i.e., ${\mathit{z}}^j={\mathit{W}}^j{\mathit{a}}^{j-1}$). Figure 2 depicts the operational principle of a crossbar array. Let us first consider the memristor in its linear zone, where it can be modeled simply as a resistor of conductance value g (adjustable) so that the memristor current is $i=g·v$ when the voltage v is applied. By scaling this to a crossbar of the size $M\times N$ and arranging the conductance values in the matrix $\mathit{G}$, we have $\mathit{i}=\mathit{G}\mathit{v}$ with $\mathit{i}={[i_1,\dots ,i_M]}^T$ and $\mathit{v}={[v_1,\dots ,v_N]}^T$ (see Figure 2). In other words, the collected currents at the output of the crossbar array are in fact a vector-matrix multiplication between the input voltages $\mathit{v}$ and the memristor conductances in $\mathit{G}$.

Let us briefly comment on the linearity of the memristor we are considering. According to the memdiode model [23], the I–V characteristic of a memristor reads as

$$I=I_0\left(\lambda \right)sinh\left[\alpha (V-IR)\right]$$

(1)

where $I_0$ is an increasing function of the parameter $0\le \lambda \le 1$ (the memory state), R is the series resistance, and $\alpha$ is a fitting parameter. Notice that Equation (1) is an implicit equation for the current I. Let us consider two extreme cases. The first is the high-resistance state (HRS) regime (with $\lambda =0$). In this case, for low voltages, we have $sinh\left(x\right)\approx x$, and the potential drop across the series resistance can also be neglected such that

$$I=I_0\left(0\right)\alpha V$$

(2)

Second, for the low-resistance state (LRS) (with $\lambda =1$), the difference is that the potential drop across R cannot be disregarded, and so

$$I=I_0\left(1\right)\alpha (V-IR)$$

(3)

which can be solved as

$$I=\frac{I_0\left(1\right)\alpha }{1+I_0\left(1\right)\alpha R}V$$

(4)

The linear regime of the memristor corresponds to a case in between these two extreme situations so that the corresponding conductance reads as

$$G\left(\lambda \right)=I_0\left(\lambda \right)\alpha (1-\lambda )+\frac{I_0\left(\lambda \right)\alpha }{1+I_0\left(\lambda \right)\alpha R}\lambda$$

(5)

which is independent of the voltage (i.e., it behaves as a simple resistor). This is the regime we are considering in our paper.

From an FFNN application point of view, the weights in $\mathit{W}$ in the software model are equivalent to the conductances in $\mathit{G}$. Usually, both positive and negative weights are represented, even when we consider only two possible values as in binary neural networks [16]. We may add a third possible value, a zero, as in ternary networks so that a particular input or activation does not affect the net outputs. As shown below, this ternary option does not modify the proposed hardware architecture (based on two crossbar arrays), as the zero weight is built naturally by combining the same conductance levels with opposite polarization. We are also exploiting the advantage of having a higher granularity when compared with its binary counterpart, as proven in [17]. Note, however, that some differences between the software model (i.e., complementary metal–oxide–semiconductor (CMOS)-based) and the memristor-based model arise. We next list the considerations in this paper:

• We need to transform the output currents at the crossbar array to voltages by means of I-to-V converters. The scale factor of the I-to-V converters is defined as $a_{I2V}$.
• The input voltages to the different layers shall be in the linear zone of the memristor (i.e., in the range $[0,V_{max}]$). Therefore, we need to scale both the inputs and the activations, because these are the inputs to the next network layers.
• We use the sigmoid as the nonlinear function f, which ranges from 0 to 1. Therefore, a scale factor of an amplitude equal to $V_{\max }$ is required.
• Memristors are set to either LRS, where the conductance is set to $g_H$, or HRS, where the conductance is set to $g_L$ (ON/OFF).
• Since the conductance values are strictly positive, a single crossbar array cannot emulate both positive and negative weights, as we have in the software model. To overcome this, we need a second crossbar array that considers the negative weights as depicted in Figure 3. Equivalently, the value of each weight in the software model $w_{m,n}^j$ is emulated by the combination $g_{m,n}^{+,j}-g_{m,n}^{-,j}$, where the superindexes + and − distinguish the first and second crossbar arrays at the jth layer, respectively.
• The memristors are programmed ex situ; that is, we first compute in the software the weights of the memristor-based neural network (considering non-idealities), and once obtained, we fix the conductances in the memristors. From that moment on, the crossbar arrays remain unchanged.
• The memristors are programmed to either $g_L$ or $g_H$, but the conductance values actually written add a random additive component. In particular, $g_{m,n}^{j,\pm }\in \{g_L+n_{g_L,m,n}^{j,\pm },g_H+n_{g_H,m,n}^{j,\pm }\}$, where $n_{g_L,m,n}^{j,\pm }\sim \mathcal{N}(0,{\sigma }_{g_L}^2)$ and $n_{g_H,m,n}^{j,\pm }\sim \mathcal{N}(0,{\sigma }_{g_H}^2)\forall m,n$. ${\sigma }_{g_L}^2$ and ${\sigma }_{g_H}^2$ represent the variances of the conductance in the HRS and LRS, respectively. We assume there are uncorrelated random additive components among the memristors.
• We consider $g_H-g_L$ to emulate the positive weight, say $+1$, $g_L-g_H$ to emulate the negative weight, say $-1$, and $g_L-g_L$ to emulate the null weight.

  Table 1. Set-up of memristors in ${\mathit{G}}^{j,+}$ and ${\mathit{G}}^{j,-}$ and corresponding ternary weights.

  ${\mathit{g}}_{\mathit{m},\mathit{n}}^{\mathit{j},+}$	${\mathit{g}}_{\mathit{m},\mathit{n}}^{\mathit{j},-}$	Ternary Weight
  $g_H$	$g_L$	$+1$
  $g_L$	$g_L$	0
  $g_L$	$g_H$	$-1$

  shows the set-up of the memristors in the positive and negative crossbar arrays and the corresponding weights. Alternatively, the null weight can be $g_H-g_H$, too. Note that our first option reduces the current and thus the power consumption.

The goals in this work are the following:

• To adjust the conductance values in  ${\mathit{G}}^{j,+}$ and ${\mathit{G}}^{j,-}$ (i.e., to decide which memristors are set to $g_H$ and which are set to $g_L$);
• To adjust the value of $g_H$, taking into account that memristors can be configured in the range $g_H\in [g_H^{min},g_H^{max}]$. Note that all the memristors are programmed to the same $g_H$ value;
• To adjust the value of $a_{I2V}$;
• To consider conductance randomness in the training process.

Note that we consider devices operating in the linear regime (i.e., in the low-voltage region), and thus nonlinearities in the I–V characteristic can be disregarded [23]. Beyond this point, the conductance of the devices may change as we move to the programming region, which is out of the scope of this work. Aside from that, line resistance, which does not affect the linearity of the devices, may also be considered, and the synaptic weights probably need to be recalculated because of the parasitic potential drops. If the devices operate in the low-voltage regime and the array is not too large, these voltage drops can be disregarded as well. This ultimately depends on the integration technology.

The next section describes the algorithm developed for the ex situ training of the memristor-based FFNN.

3. Proposed Algorithm

In this section, we consider the equivalent software model in Figure 1 in order to train and configure our memristor-based FFNN, depicted in Figure 3.

3.1. Training of Quantized Neural Networks

Training of the resulting quantized neural network is accomplished using the so-called backpropagation algorithm as described in [16]. The idea is simple: the forward pass in the backpropagation applies the quantization, whereas the backward pass computes the gradients as usual in order to update the weights. Stochastic and efficient optimization is accomplished by randomly shuffling the data and by training consecutively on small subsets of the data, respectively. Algorithm 1 shows the steps of the training process.

Algorithm 1 Algorithm for training a quantized network.

Input: Batch of training examples and labels     Output: ${\widehat{\mathit{W}}}^1,\dots ,{\widehat{\mathit{W}}}^J$           Initialization       1: Randomly initialize the weights ${\mathit{W}}_0^1,\dots ,{\mathit{W}}_0^J$ at the J layers in the FFNN           LOOP Process       2: for $k=1$ to $N_{\mathrm{epochs}}$ do       3:    for $t=1$ to $N_{\mathrm{mini}-\mathrm{batch}}$ do       4:       ${\widehat{\mathit{W}}}_t^j=q\left({\mathit{W}}_t^j\right)\forall j$ (q is defined in Equation (6) below)       5:       Forward propagation: compute network activations and outputs using ${\widehat{\mathit{W}}}_t^j$       6:       Backward propagation: use ${\mathit{W}}_t^j$ to compute gradients       7:       ${\mathit{W}}_{t+1}^j⟵{\mathit{W}}_t^j+\alpha \nabla {\mathit{W}}_t^j$       8:    end for       9: end for     10: return${\widehat{\mathit{W}}}_t^j$

3.2. Ternarization

We considered the following quantization function $q\left(x\right)$, which is defined as

$$q\left(x\right)=\left\{\begin{array}{cc}-g_H+g_L& x<-\Delta \hfill \\ 0& -\Delta \le x\le \Delta \hfill \\ g_H-g_L& x>\Delta \hfill \end{array}\right.$$

(6)

We considered two options to fix $\Delta$. The first one was to set it to a fixed value. The second one was to try to optimize the value of $\Delta$ according to the current weights at time t in ${\mathit{W}}_t^j$ so that $\Delta$ was updated at each iteration of the algorithm. We followed the work in [17] to adjust the value of $\Delta$ as

$${\Delta }_t=\frac{0.7}{N_{\mathrm{total}}}\sum _{j=1}^J{\mathit{1}}^T{\mathit{W}}_t^j\mathit{1}$$

(7)

where $\mathit{1}$ is the all-ones column vector and $N_{\mathrm{total}}$ is the total number of weights in the FFNN. The aim was to adapt the threshold to the current distribution of the weights. Note that ${\Delta }_t$ is the same for all network layers in our work, although different thresholds per layer could also be considered.

3.3. Adaptation of $g_H$

The proposed crossbar structure has two additional parameters to configure. Recall that we assumed an ON/OFF memristor model and that the conductances for the LRS and HRS were common to all memristors in the array. Notwithstanding, memristors can be programmed to different conductance values in the LRS. In this subsection, we develop the tuning of the conductance in $g_H$. Recall that in Algorithm 1, we configured the memristors in our network to either the LRS or the HRS, relying on backpropagation. In particular, note that the unquantized weights that are written in the memristor network, as the LRS will generally differ from $g_H$. In other words, usually we have

$$|w_{m,n,t}^j|\ne g_H-g_L\forall m,n,js.t.\left|{\widehat{w}}_{m,n,t}^j\right|=g_H-g_L$$

(8)

Therefore, we can use the values in $w_{m,n,t}^t$ to also update $g_H$ and reach a consensus value $g_H^{*}$. Consider the following update rule:

$$\begin{array}{ccc}\hfill g_H^{t+1}& =& (1-{\alpha }_{g_H})·g_H^t\hfill \\ & +& {\alpha }_{g_H}\left[g_L+\frac{1}{N_{\ne 0}^{t+1}}\sum _{j=1}^J{\mathit{1}}^T\mathcal{M}\left({\mathit{W}}_{t+1}^j\right)\mathit{1}\right]\hfill \end{array}$$

(9)

where ${\alpha }_{g_H}$ is the forgetting factor and $\mathcal{M}$ is a masking function that operates element-wise in order to consider only the weights that have influence in $g_H$ (i.e., not the null weights). When $\mathcal{M}$ is applied to a scalar in ${\mathit{W}}_{t+1}^j$, say $w_{m,n,t+1}^j$, it produces the following output:

$$\mathcal{M}\left(w_{m,n,t+1}^j\right)=\left\{\begin{array}{cc}-w_{m,n,t+1}^j,& {\widehat{w}}_{m,n,t+1}^j=-g_H+g_L\hfill \\ 0,& {\widehat{w}}_{m,n,t+1}^j=0\hfill \\ w_{m,n,t+1}^j,& {\widehat{w}}_{m,n,t+1}^j=g_H-g_L\hfill \end{array}\right.$$

(10)

Additionally, $N_{\ne 0}^{t+1}$ is the total number of weights whose quantization is different from zero at iteration $t+1$.

However, note that a single weight in the neural network, say $w_{m,n,t}^j$, once quantized, requires three elements in our hardware model to be represented: two memristors (one in ${\mathit{G}}_t^{j,+}$ and one in ${\mathit{G}}_t^{j,-}$) and an I-to-V converter. In other words, ${\widehat{w}}_{m,n,t}^j$ is represented in our physical model as $(g_{m,n,t}^{j,+}-g_{m,n,t}^{j,-})·a_{I2V}$. Furthermore, we set $a_{I2V}=1/g_H$ (assuming $1/(g_H-gL)\approx 1/g_H$ when $g_H\gg g_L$) in order to map the weights $\{-1,0,1\}$, as is the case in software-based ternary networks [17]. However, the conductance variance ${\sigma }_{g_H}^2$, which does not depend on the particular value of $g_H$, now plays an important role, and the best choice is to set $g_H$ to the largest value allowed. Note that after division by $g_H$ in the I-to-V converter, the resulting conductance variance is also downsized.

In short, the discussion above is to point out that the best strategy is to set $g_H$ as large as possible and then fine-tune our memristor-based neural network by adjusting the gain in the I-to-V converter, as we show next.

3.4. Adaptation of $a_{I2V}$

Let us consider that each output at the crossbar array could be adjusted separately (i.e., we have $a_{I2V,n}^j$). In this case, it is not complicated to compute the gradients for these parameters. It is similar to the weight gradients in backpropagation. For example, consider the scores ${\mathit{z}}^J={\mathit{a}}_{I2V}^J\odot {\mathit{W}}^J{\mathit{a}}^{J-1}$, where ⊙ stands for the Hadamard product at the output layer of the neural network. If we train it using cross-entropy (assuming a classification task) (i.e., $L=-{\sum }_{i=1}^{N_J}{t}_{i}log{y}_i$, where $N_J$ is the number of classes, $t_i$ (0 or 1) are the targets and $y_i$ are the network outputs), the gradients are found as follows:

$$\frac{\partial L}{\partial a_{I2V,i}^J}=\frac{\partial L}{\partial z_i^J}·\frac{\partial z_i^J}{\partial a_{I2V,i}^J}=(y_i-t_i)·{\left({\left[{\mathit{W}}^J\right]}_i\right)}^T{\mathit{a}}^{J-1}$$

(11)

where ${\left[\mathit{X}\right]}_i$ selects the ith row of matrix $\mathit{X}$.

Let us analyze the effect of this gradient in the network, depicted in Figure 4. Assume that $t_i=1$ (so the current example belongs to the ith class). Unless we get a perfect classification, $y_i<1$ and the first term $(y_i-t_i)$ of the gradient will be negative. Therefore, if ${\left({\left[{\mathit{W}}^J\right]}_i\right)}^T{\mathit{a}}^{J-1}$ is positive (i.e., we are at the positive side of the sigmoid or softmax), the gradient is negative, and $a_{I2V,i}^J$ should be increased according to Equation (11). The effect is to shrink the sigmoid or softmax in order to increase the value of $y_i$. If ${\left({\left[{\mathit{W}}^J\right]}_i\right)}^T{\mathit{a}}^{J-1}$ is negative, we are on the negative x-axis of the sigmoid or softmax, and the update of $a_{I2V,i}^J$ stretches the curve. This increases the value of $y_i$ and therefore diminishes the classification error. The reader can refer to Figure 4 for a graphical visualization of the discussion above. The analysis for $t_i=0$ is similar and not included here for the sake of brevity.

Having separate conversion gains at all crossbar outputs that are individually adapted is a real possibility. However, since we assume a common converter value, we must build a consensus gradient from all individual gradients such that

$$\frac{\partial L}{\partial a_{I2V}}=\frac{1}{N_{\mathrm{out}}}\sum _{j=1}^J{\mathit{1}}^T\frac{\partial L}{\partial {\mathit{a}}_{I2V}^j}$$

(12)

where $N_{\mathrm{out}}={\sum }_{j=1}^J{N}_j$ (i.e., the total number of outputs in the J layers of the FFNN). We can now apply gradient-descent-based solutions to optimize $a_{I2V}$.

Another option is to simply consider $a_{I2V}$ as a hyperparameter of the neural network (it is a scalar value) and optimize.

3.5. Including Robustness in Perturbed Conductances

The last issue we consider is the perturbation of the conductances that are written to the memristors; that is, we want to set the memristor to a conductance level $g_L$ or $g_H$, but the level we actually achieve differs by a Gaussian perturbation term (zero-mean and variances ${\sigma }_{g_L}^2$ and ${\sigma }_{g_H}^2$, respectively).

In order to cope with this physical impairment, we adopted an approach that resembles the training of quantized networks. Specifically, in the backward pass of backpropagation, we added a Gaussian term to the weights. The variance of that random contribution was set to ${\sigma }_{g_P}^2$, which is a hyperparameter of the network. In other words, we considered the following approach (in algorithmic style). This step substitutes step 7 in Algorithm 1.

${\mathit{W}}_{t+1}^j⟵{\mathit{W}}_t^j+\alpha \nabla {\mathit{W}}_t^j+\mathcal{N}\left(0,{\sigma }_{g_P}^2\right)$

The approach has a well-established foundation that connects to the regularization methods in neural networks. Primarily used in the context of recurrent neural networks, as described in [24] (Ch. 7.5), noise injection (i.e., adding random values to the weights) adds robustness to the network in the sense that the model learned is somehow insensitive to small variations in the weights. In other words, our approach can be interpreted as a form of regularization.

4. Experimental Results

In this section, we experimented with the proposed ternary network in order to evaluate the effects of the different adaptation mechanisms (conductance at the LRS and the conversion factor at the I-to-V stage), as well as the effect of quantizing the weights and the incorporation of weight variability during training. We considered two different datasets widely employed as benchmark datasets in machine learning: the Modified National Institute of Standards and Technology(MNIST) dataset [25] and the fashion MNIST dataset [26]. Both datasets consist of grayscale images of 28 × 28 pixels. The former contains images of handwritten numbers (from 0 to 9), whereas the latter also contains also different types (or classes) of images all related to clothes (e.g., t-shirts, pullovers or sandals, among others). In both cases, an 80–20 random split for training and testing was conducted.

In terms of neural network architecture, we considered an FFNN with two hidden layers of a size 1000 units/neurons. Taking into account 784 (=28 × 28) values at the input layer and 10 output classes, the whole architecture was 784–1000–1000–10, with a total of 7.85 G weights/parameters to be trained (including bias terms) in a full-software implementation and 15.7 G memristors to be set at either the LRS or HRS in the memristor-based neural network implementation. Training and evaluation were performed by means of Python programming using the Tensorflow library for deep neural networks [27]. The memristors were modelled in Python and Tensorflow according to the assumptions in Section 2.

Table 2. Electrical parameters in the memristor-based neural network.

Parameter	Value or Range
$V_{max}$	0.2 V
$g_L$	1
$g_H$	[14, 140]
${\sigma }_{g_L}$	1
${\sigma }_{g_H}$	1–10

summarizes the electrical parameters considered in our experiments. We considered values that were in agreement with the state of the art of the memristive technology [10,28], but we also took into account larger conductance deviations in order to accommodate other fabrication technologies. Our goal here was to test the practical importance of synthesizing reliable devices in terms of conductance fluctuations. The conductance values are always relative to $G_0$, the quantum conductance.

Finally, ternarization was applied with the adaptive threshold in Equation (7), and the performance metric employed was classification accuracy, which measured the number of examples (images in this case) correctly classified with respect to the total number of images (i.e., it was the percentage of images correctly classified).

4.1. Adaptation of $g_H$

Our first experiment dealt with the adjustment of the conductance value for the LRS (i.e., $g_H$). In Figure 5 and Figure 6, we considered violin plots that showed the distribution of accuracies obtained after 1000 realizations. Remember that memristor conductances incorporate a random term (i.e., $g_{m,n}^{+,j}=g_H+n_{g_H,m,n}^{+,j}$ and $g_{m,n}^{-,j}=g_L+n_{g_L,m,n}^{-,j}$). Although the weights computed during training remained unchanged, their mapping to memristor conductances changed from one realization to another due to the random term. That aside, we considered here $a_{I2V=1/g_H}$, the threshold used for the ternarization of the weights set as in Equation (7), and ${\sigma }_{g_P}=0$. Figure 5 considers ${\sigma }_{g_H}=1·G_0$, and Figure 6 considers ${\sigma }_{g_H}=10·G_0$.

As we can appreciate in Figure 5, where the conductance perturbations were moderate, the average accuracy was above 95% with all the tested adjustments of $g_H$. However, the distribution of accuracy values was spread out significantly more for the case $g_H=14$ (ranging from 0.845 to 0.976), whereas the dispersion diminished for the case of $g_H=30$ (ranging from 0.939 to 0.977) and practically vanished for $g_H=60$ (ranging from 0.965 to 0.978) and more notably for $g_H=140$ (ranging from 0.976 to 0.979).

Figure 6 involves experiments with a severe conductance perturbation. In this case, the classification task was more complex, and with equal network configuration, the performance dropped. We appreciated the dispersion in the accuracy distributions for all cases of $g_H$, although the dispersion tended to reduce as $g_H$ increased. In this best case, the accuracy ranged from 0.551 to 0.825, so the difference between the max and min value was 0.274. In this application, it is important to note the mean values for the accuracy. For the two lowest conductance values, the mean accuracy was 25.8% for $g_H=14·G_0$ and 41.9% for $g_H=30·G_0$. This value grew up to 61.1% for $g_H=60$ and to 73% for $g_H=140$.

In conclusion, both experiments confirmed that $g_H$ should be adjusted to the highest possible value (depending on the available technology) in order to achieve the best possible performance.

4.2. Adaptation of $a_{I2V}$ and Robustness to Perturbed Conductance Values

In Figure 7, we tested how sensitive the classification accuracy was to adjustment of the value in $a_{I2V}$ and to the weight perturbance introduced during training (i.e., ${\sigma }_{g_P}$), assuming $g_H=140·G_0$. We considered both datasets under study (handwritten digits and fashion MNIST), and we plotted the classification accuracy as a function of $g_h·a_{I2V}$, testing different combinations of ${\sigma }_{g_H}$ and ${\sigma }_{g_H}$, particularly ${\sigma }_{g_H}\in \{1·G_0,5·G_0,10·G_0\}$ and ${\sigma }_{g_P}\in \{0,1·G_0,5·G_0,10·G_0,20·G_0,50·G_0\}$. As we can appreciate in the figure, setting $a_{I2V}=1/g_H$ (i.e., $a_{I2V}·g_H=1$) gave us a particularly good initial adjustment. In the applications tested, the plots show that the performance could be just slightly increased by choosing the optimal value of $a_{I2V}$, as long as the sensitivity around the initial adjustment was low. Note that the perturbance introduced during training (i.e., the value in ${\sigma }_{g_P}$) had a larger influence on the classification accuracy (i.e., the different accuracy curves became more separated), especially for the fashion MNIST dataset when ${\sigma }_{g_H}=5·G_0$ and ${\sigma }_{g_H}=10·G_0$. Note also that in general, the higher ${\sigma }_{g_H}$ was, the more variation in performance we observed. As a rule of thumb, setting ${\sigma }_{g_P}$ to a value in the range $[{\sigma }_{g_H},5·{\sigma }_{g_H}]$ provided a proper adjustment.

4.3. Comparison with the Software-Based Neural Network

We then tested how a memristor-based neural network (MBNN) compared to a software-based neural network (SBNN). For this occasion, we took into account ternarization of the weights as well as mitigation of the conductance perturbations and optimization of $a_{I2V}$. Aside from the reference network architecture (i.e., 784–1000–1000–10), we also considered 784–500–500–10 and 784–100–100–10 for the MBNN and SBNN models. The results reported the empirical cumulative density functions (ecdf). Note that the SBNN suffered no perturbation once the weights were fixed, but the performance slightly varied due to the random initialization of weights in training, too. In order to reflect this issue, we then considered 1000 realizations in total for each model containing 10 different training processes. In other words, the same set of weights was used to perform 100 inferences. Note that in the SBNN, all inferences that used the same set of weights produced the same results, whereas in the MBNN, this was not the case due to conductance fluctuations. We considered here the classification of the fashion MNIST dataset, which is a more complex task than handwritten digit classification, assuming $g_H=140·G_0$ and ${\sigma }_{g_H}=10·G_0$.

In Figure 8, we next compare the following methods: (1) SBNNs with 1000, 500 and 100 units in the two hidden layers; (2) an MBNN (1000 units in the hidden layers) with ${\sigma }_{g_P}=0$, $a_{I2V}=1/g_H$ (i.e., we did not consider conductance fluctuations in training), an MBNN with ${\sigma }_{g_P}=10·G_0$, $a_{I2V}=1/g_H$ (i.e., a default set-up assuming fluctuations in training) and a fine-tuned MBNN (in this case requires increasing ${\sigma }_{g_P}$ to $50·G_0$); and (3) the fine-tuned MBNN version with 500 and 100 units in the hidden layers.

The results essentially show two issues when we considered 1000 units at each hidden layer. First of all, there was the importance of considering memristor fluctuations during training. Note the spreading in the ecdf for ${\sigma }_{g_P}=0$, which had a maximum value of 0.8724 and a minimum value of 0.6274 (i.e., the gap was 0.245). This gap significantly reduced to 0.042 in the case of the default set-up and practically vanished in the tuned MBNN and also the SBNN. Second, a properly tuned MBNN achieved a performance that was similar to the SBNN counterpart. If we look at the worst performance in all the set-ups, the SBNN archived an accuracy of 0.8788, the MBNN with ${\sigma }_{g_P}=0$ yielded 0.6274 (a 28.6% reduction with respect to the SBNN), and the MBNN with ${\sigma }_{g_P}=50·G_0$ obtained 0.8905 (a 1.3% increase with respect to the SBNN). This slightly better performance might have been due to the regularization effect produced when we included robustness to perturbed conductance values by means of ${\sigma }_{g_P}$. For the cases of 500 and 100 hidden units per layer, the MBNN performed close the SBNN for 500 units and suffered a reduction of about 3% in accuracy for the case with 100 hidden units. Therefore, weight quantization affected the performance more as the complexity of the model was further constrained.

4.4. Summary and Extension of Results

To summarize the results so far, we saw that the performance in general depended on the task complexity, the network configuration as well as on the memristor quality, where the larger the $g_H$ and the lower the ${\sigma }_{g_H}$, the better. Regarding task complexity, Figure 7 plots the results of the exact same models applied to two different tasks. In the top row, the less complex task showed less variability among models such that a proper adjustment of ${\sigma }_{g_P}$ was less critical. In the bottom row, the more complex task showed more variability and required a good adjustment of ${\sigma }_{g_P}$. In order to complete our analysis, now with a lower quality memristor, we could reproduce in Figure 9 the same experiments while considering $g_H=14·G_0$ instead of $g_H=140·G_0$. As we can appreciate in the figure, the classification accuracy in the different models was far more sensitive to the proper adjustment of ${\sigma }_{g_P}$. The worst performing models in the classification of handwritten digits (top row) then achieved accuracies around or below 20%, whereas the worst accuracy in Figure 7 was above 75%. Something similar occurred in the classification of clothes; the worst performing models achieved values around 20% whereas in Figure 7, the worst performance was above 65%.

Finally, we tested our memristor-based solution as part of a convolutional neural network (CNN) implementation applied to the classification of the fashion MNIST dataset. The input in this case was grayscale images of 28 × 28 pixels (i.e., 2D data). The configuration of our CNN was as follows:

• 2D convolutional layer with 32 filters, 3 × 3 kernels and rectified linear unit (ReLU) activation;
• 2D max pooling 2 × 2 layer;
• 2D convolutional layer with 64 filters, 3 × 3 kernels and ReLU activation;
• 2D max pooling 2 × 2 layer;
• 2D convolutional layer with 128 filters, 3 × 3 kernels and ReLU activation;
• Flatten layer (1152 values at output);
• Fully connected layer (1000 values at output);
• Fully connected layer (1000 values at output);
• Fully connected layer (10 values at the output to identify each of the 10 classes in the dataset).

Convolutional, max pooling and flatten layers were implemented in the software. This first block transformed each image into 1152 positive values at the output of the flatten layer (1D). The second block embraced the fully connected layers and was identical to the FFNN tested so far, except for the number of input values (768 before vs. 1152 now). This second block was implemented both in the software and using the proposed memristor-based neural network. In the latter case, the outputs at the flatten layer were scaled to fit the range $[0,V_{max}]$. Note that the memristors could also be considered in the first block, but this introduces additional complexity in so far as more peripheral circuitry is required. This point is beyond the scope of the paper. In Figure 10, we reproduced the results in Figure 8 using the described CNN approach.

The results show that the performance, in terms of prediction accuracy, increased for all the configurations tested with respect to the FFNN approach. This was due to the high-level features processed at the convolutional and max pooling layers. The second observation is that increasing the value of ${\sigma }_{g_P}$ gave robustness to the system (i.e., the accuracy values were less spread out). This result is coherent with the results obtained so far. Finally, we observed that the MBNN with a proper configuration was close in performance to the SBNN. These preliminary results encourage us to explore the application of memristors to more complex neural network architectures.

5. Conclusions

In this paper, we analyzed the implementation of deep neural networks using crossbar arrays of memristors, and more specifically, we considered the case where these devices can be configured in only two different states: a low-resistance state (LRS) and a high-resistance state (HRS). The natural usage of crossbar arrays in the context of neural networks is in performing vector-matrix multiplications in an analog fashion (i.e., by adding currents), thus reducing the power consumption and computational time. Our approach aims at emulating ternary neural networks, which sets the weights in the neural network to a value in the range of $\{-1,0,1\}$. In order to achieve this behavior, we need to implement two crossbar arrays for each feedforward layer in the network (i.e., one to represent the positive weights and the other one to represent the negative weights). Additionally, some other adaptation issues in relation to software-based neural networks arise: (1) the currents at the output of the crossbar arrays have to be converted to voltages for the next stage, resulting in a conversion factor that can be potentially tuned to boost network performance and (2) memristor device experiment conductance fluctuations that also impinge on performance. Taking these issues into account, we designed an algorithm to train the weights in the network and later map these weights to the network, where memristors are programmed to either the LRS or the HRS.

The results show that the proposed system design and offline training method represent a real alternative to the traditional software-based (i.e., CMOS-based) neural networks. The lessons learned in this work are as follows: (1) the higher the conductance of the memristor in the LRS, the better performance we can achieve; (2) the conversion factor that maps the output currents at one layer to input voltages at the next layer can be fine-tuned, but it is not a sensitive parameter; and (3) it is very important to consider mitigation of the conductance variability, as performance is very sensitive to this. In our experiments, we achieved accuracies that were similar to the software-based counterpart, but without considering conductance variability during training, we observed large gaps in terms of classification accuracy between the worst realizations. This gap could be above 50% in the 10-class classification tasks (handwritten digits and fashion MNIST data) we tested.

Future work could consider additional hardware issues such as nonlinearity, stochasticity, varying maxima, asymmetry between increasing and decreasing responses, non-responsive devices at low or high conductance, mixed time-varying delays or the sneak-path problem in crossbar arrays [10,11,12,13,14]. We also need to evaluate the performance using more complex and widely used neural network models, such as convolutional or recurrent networks. The preliminary results have been presented for a CNN here, showing the potential of memristor-based approaches.