97.6% array area reduction, ReRAM computation-in-memory with hyperparameter optimization based on memory bit-error rate and bit precision of log-encoding simulated annealing

In the internet of things (IoT) era, solving combinatorial optimization problems effectively is required in various situations, such as logistics, finance, and traffic. However, it is difficult to solve combinatorial optimization problems in the certain period time by present Neumann-architecture computers because the number of combinations explodes as factors increase. ^1,2) Recently computer architectures inspired by the Ising model and simulated annealing (SA) have been studied. ^3–7) The Ising model represents the behavior of a magnetic field. Combinatorial optimization problems can be solved by finding the ground state of energy given by Hamiltonian in the Ising model. ^8,9) SA is an algorithm to search the optimal answer effectively. In SA, energy is repeatedly and sequentially calculated as temperature decreases. The optimal answer is obtained where energy reaches the global minimum state. ^10–12) However, the large amount of calculation and their energy consumption are problems.

Thus, this paper proposes small array area and memory error tolerant resistive random access memory (ReRAM)-based computation-in-memory (CiM) with hyperparameter optimization based on bit-error rate (BER) and bit precision of log-encoding SA, as shown in Fig. 1. ReRAM CiM for IoT edge devices has attracted much attention due to its energy efficiency, high throughput, and scalability. ^14,15) ReRAM CiM can solve combinatorial optimization problems by calculating energy using multiply-accumulate (MAC) operations. ¹⁵⁾ In this paper, the knapsack problem is discussed, which is one of the combinatorial optimization problems and is also used in financial portfolios. In the case of the knapsack problem, the array area of the conventional ReRAM CiM becomes correspondingly larger because the number of spins for knapsack capacity linearly increases. Therefore, to reduce the array area, log-encoding SA is proposed. ^3,16)

Zoom In Zoom Out Reset image size

In terms of a ReRAM device, on the one hand, a ReRAM device has reliability issues because ReRAM causes bit-error at high Set/Reset cycles. ^15,17,18) On the other hand, as for bit-error in SA, it has been reported that searching the optimal answer by controlling the device noise properly is more accurate. ¹³⁾ Therefore, by co-design of a ReRAM device and log-encoding SA, the influence of the asymmetric ReRAM error characteristics and bit precision on SA is analyzed. Furthermore, this paper shows that the proposed ReRAM CiM improves success probability, which is defined as the probability of obtaining the optimal answer, from four aspects: (1) storing "0" in reliable state in ReRAM CiM, (2) adjusting random flips and a cooling parameter, (3) delaying Error Injected (EI) iteration, and (4) hyperparameter optimization based on BER and bit precision. ¹⁶⁾ As for applied technology that requires extremely high accuracy, hyperparameter optimization can be effective.

This paper is organized as follows. Section 2 describes log-encoding SA to reduce the array area of ReRAM CiM. Section 3 describes the co-design of a ReRAM device and log-encoding SA. Section 4 analyzes ReRAM error injection to SA with success probability. Finally, Section 5 summarizes this work.

To solve the knapsack problem by MAC operation in ReRAM CiM, first, the knapsack problem is transformed as Hamiltonian. Next, the formulated Hamiltonian is converted to quadratic form and then mapped to ReRAM CiM. In other words, the array area of ReRAM CiM depends on the size of the Hamiltonian in quadratic form. To reduce the array area, the size of the Hamiltonian in quadratic form must be reduced. Thus, log-encoding is proposed for the Hamiltonian. Hamiltonian $H$ consists of 2 parts, a cost function ${H}_{A}$ and a penalty function ${H}_{B}$ in the following Eq. (1).

$$\begin{eqnarray}&&H={H}_{A}+{H}_{B}.\end{eqnarray} \tag{ 1 }$$

The cost and penalty functions of the proposed log-encoding are calculated by the following Eqs. (2) and (3), respectively, where $W$ is knapsack capacity, ${v}_{\alpha }$ is value of each item, ${w}_{\alpha }$ is weight of each item, $\sigma ,$ $\mu$ are hyperparameters. ${x}_{\alpha }$ is for items (${x}_{\alpha }\in \left\{0,1\right\}$), where 0 means that item is unselected and 1 means that item is selected. ${y}_{i}$ is for extra variable for total weight (${y}_{i}\in \left\{0,1\right\}$). ^3,16)

$$\begin{eqnarray}&&{H}_{A}=-\sigma \displaystyle \sum _{\alpha =1}^{N}{v}_{\alpha }{x}_{\alpha },\end{eqnarray} \tag{ 2 }$$

$$\begin{eqnarray}&&{H}_{B}=\mu {\left(W-\displaystyle \sum _{i=1}^{\unicode{x0230A}{\mathrm{log}}_{2}W-1\rfloor +1\unicode{x0230B}}{2}^{i}{y}_{i}-\displaystyle \sum _{\alpha =1}^{N}{w}_{\alpha }{x}_{\alpha }\right)}^{2}.\end{eqnarray} \tag{ 3 }$$

In the conventional linear-encoded Hamiltonian, although the cost function ${H}_{A}$ is the same as the proposed log-encoded Hamiltonian shown in Eq. (2), the penalty function ${H}_{B}$ is the following Eq. (4), where $\lambda$ is hyperparameter. ¹⁵⁾

$$\begin{eqnarray}&&{H}_{B}=\lambda {\left(1-\displaystyle \sum _{i=1}^{W}{y}_{i}\right)}^{2}+\mu {\left(\displaystyle \sum _{i=1}^{W}i{y}_{i}-\displaystyle \sum _{\alpha =1}^{N}{w}_{\alpha }{x}_{\alpha }\right)}^{2}.\end{eqnarray} \tag{ 4 }$$

To map to ReRAM CiM, Hamiltonian Eq. (1) is converted to quadratic form Eq. (5).

$$\begin{eqnarray}&&H={q}^{T}Qq\end{eqnarray} \tag{ 5 }$$

$q$ represents spins, which consist of ${x}_{\alpha }$ and ${y}_{i}.$ QUBO (Quadratic Unconstrained Binary Optimization) matrix $Q$ is shown in Eq. (6). ^19,20)

$$\begin{eqnarray}Q=\left[\begin{array}{ccc}\mu {w}_{1}^{2}-2\mu W{w}_{1}-\sigma {v}_{1} & \cdots & {2}^{1+\unicode{x0230A}{\mathrm{log}}_{2}{\rm{W}}-1\unicode{x0230B}+1}\mu {w}_{1}\\ 0 & \cdots & \vdots \\ \vdots & \ddots & \vdots \\ 0 & \cdots & {2}^{2+\unicode{x0230A}{\mathrm{log}}_{2}W-1\unicode{x0230B}+1}\mu \end{array}\right].\end{eqnarray} \tag{ 6 }$$

In the conventional linear-encoding as shown in Eq. (4), as knapsack capacity $W$ becomes larger, the number of ${y}_{i}$ linearly increases. As a result, the array area correspondingly becomes larger. Thus, by applying log-encoding to the penalty function Eq. (3), the number of ${y}_{i}$ can decrease and be $\unicode{x0230A}{\mathrm{log}}_{2}W-1\unicode{x0230B}+1.$ Figure 2 describes an example with 10 items and the knapsack capacity of 100. In the proposed log-encoding, only 7 spins for ${y}_{i}$ are required because the decimal number of 100 can be represented by 7 binaries. The proposed spins are 17 whereas the conventional spins are 110. Figure 3 shows the array area comparison between the proposed and the conventional ReRAM CiM. ${q}^{T},$ $Q$ and $q$ of the Hamiltonian in quadratic form are assigned to the bit-line (BL), the conductance of ReRAM, and the word-line (WL) of ReRAM CiM, respectively. ^21,22) The proposed ReRAM CiM is 17 WLs $\times$ 17 BLs that reduces the array area by 97.6%, compared with the conventional ReRAM CiM, 110 WLs $\times$ 110 BLs.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Figure 4 shows the result of Set/Reset operations on 8 Kbit cells of a 40 nm TaO_X -based ReRAM device, ^23,24) measured at room temperature. Figure 4(a) shows the cell current distribution of low resistance state (LRS) and high resistance state (HRS) in cumulative probability plots. The cell current distributions of HRS are almost unchanged through 10⁶ set/reset cycles. However, in LRS, tail bits increase as set/reset cycles increase. Figure 4(b) shows BER of HRS and LRS. Although BER of HRS stays at low, BER of LRS increases after 10⁴ set/reset cycles. ReRAM has the asymmetric error characteristics.

Zoom In Zoom Out Reset image size

To calculate the energy of the Hamiltonian by ReRAM CiM, the QUBO matrix (Q) of the Hamiltonian in quadratic form is mapped into ReRAM CiM (G) according to the bit precision of ReRAM by the following Eq. (7). ¹⁶⁾

$$\begin{eqnarray}&&\begin{array}{l}G\left({G}_{ij}\right)\\ =\,Q\left({a}_{ij}\right)\times \displaystyle \frac{{2}^{M}-1}{\max \left(\left|{a}_{ij}\right|\left|1\leqslant i,j\leqslant N+\unicode{x0230A}{\mathrm{log}}_{2}W-1\unicode{x0230B}\right.+1\right)},\end{array}\end{eqnarray} \tag{ 7 }$$

Where M is bit precision, N is number of items, and W is the knapsack capacity. The elements of the QUBO matrix, "0" or "1", are stored in HRS or LRS of ReRAM. This means that the asymmetric ReRAM error characteristics influences energy calculation, and hence success probability. Thus, "0" and "1" error injection is proposed to reproduce the asymmetric ReRAM error characteristics to SA by changing the ratios of "0" error to "1" error. ¹⁶⁾

Figure 5 describes "0" and "1" error injection. The ratio of "0" error to "1" is 1 to 0 in Case 1 whereas the ratio is 0 to 1 in Case 2. In an example with BER of 10%, 10% of "0" error is injected in Case 1 while 10% of "1" error is injected in Case 2. In "0" error injection, randomly selected 0 is flipped to 1 in accordance with BER by representing the element of ReRAM CiM as a binary number. Similarly, in "1" error injection, randomly selected 1 is flipped to 0.

Zoom In Zoom Out Reset image size

Figure 6 describes the QUBO matrix with Case 1 and Case 2 of "0" and "1" error injection. The proportion of elements, "0" and "1", in the QUBO matrix also influences success probability because of the asymmetric ReRAM error. All the elements of the QUBO matrix below the main diagonal are 0 because the QUBO matrix is the upper triangular matrix. Case 1 demonstrates that 0 is stored in LRS because LRS causes error. For a similar reason, Case 2 demonstrates that 0 is stored in HRS.

Zoom In Zoom Out Reset image size

4.1. "0" and "1" error injection

Figure 7 shows the colourmaps of the proposed log-encoding SA results with "0" and "1" error injection. Success probability is defined as the probability of obtaining the optimal answer in 1000 trials of a knapsack problem. ^25,26) The ratio of "0" error to "1" error is Fig. 7(a) 1:0 (Case 1), Fig. 7(b) 0.7:0.3, Fig. 7(c) 0.5:0.5, Fig. 7(e) 0.3:0.7, and Fig. 7(f) 0:1 (Case 2). In the order from Fig. 7. (a) to Fig. 7(f), the ratio of "0" error decreases whereas the ratio of "1" error increases. It can be seen that success probability increases as "0" error decreases. In other words, storing "0" in HRS, which is more highly reliable than LRS, increases success probability. Furthermore, all the colourmaps show that higher bit precision tolerates higher BER. Conversely, if BER is lowered, bit precision can be reduced.

Zoom In Zoom Out Reset image size

Figure 8 shows the success probability of the different ratios of "0" and "1" error injection with Fig. 8(a) 10 bit, Fig. 8(b) 7 bit, and Fig. 8(c) 4 bit precision. As shown in Fig. 8, Case 1 has the highest success probability and Case 2 has the lowest success probability regardless of bit precision. It means that success probability increases as the ratio of "0" error decreases. Acceptable probability is defined as more than 90%. As shown in Figs. 8(a) and 8(b), in 10 bit and 7 bit precision, BER of 10% is allowed in Case 1. Case 1 increases the acceptable BER by 10 times, compared with Case 2 because Case 2 allows only 1%. In 4 bit precision, as shown in Fig. 8(c), in Case 1, no error is allowed while in Case 2, up to 7 bit precision is allowed.

Zoom In Zoom Out Reset image size

Figure 9 shows the comparison between the success probability of Case 1 and Case 2 with the different BER. As shown in Fig. 9(a), in the case of BER of 10%, even if the bit precision is reduced to 4 bit, the success probability reaches the acceptable probability in Case 1. As for Case 2, up to 7 bit precision is allowed. In the case of the BER of 5%, the acceptable bit precision is 4 bit in Case 1 whereas no error is allowed in Case 2 [Fig. 9(b)]. As shown in Fig. 9(c), when BER increases to 10%, no error is allowed in Case 2, and in Case 1 the acceptable bit precision is 5 bit, 1 bit less than the acceptable bit precision of BER of 5%. The largest bit between the acceptable precision of Case 1 and Case2 is 6 bit in the case of BER of 5%.

Zoom In Zoom Out Reset image size

As mentioned above, Case 2 has better success probability than Case 1 in terms of the acceptable BER and bit precision. Considering the ReRAM device in SA, it is better that 0 is stored in HRS, which is more reliable, and 1 is stored in LRS, which causes error due to the increases of set/reset cycles. For this reason, in the following sections, Case 2 is studied.

4.2. Random flip and cooling parameter

In SA, the transition of energy is sometimes stuck in a local minimum. In this case, the optimal answer is not achieved because the spins ${x}_{i}$ obtained by the local minimum energy becomes the answers. Thus, a random flip is introduced to prevent the spins ${x}_{i}$ from staying at local minimums. ²⁷⁾ As for random flips, the spins ${x}_{i}$ are randomly flipped k-times in each iteration. The number of flips k is the same number of items, and it is allowed that the same spin is repeatedly flipped. However, under these conditions, the SA process takes longer because the number of flips increases with the number of items. Therefore, to reduce the SA process, the number of random flips is reduced by changing k. Figure 10(a) shows the illustration of random flips in a knapsack problem of 10 items with 10-time and 6-time random flips. In addition to this, the iteration of SA can be reduced by changing cooling parameter $\alpha .$ ²⁸⁾ A smaller number of cooling parameters simulates a more rapid annealing because the temperature of SA decreases more rapidly. In other words, iteration is reduced by reducing the cooling parameter. Figure 10(b) shows the SA results of 10-time and 6-time random flip with the different cooling parameter $\alpha$ in Case 2. When the cooling parameter is 0.998, iteration is 1956, and when the cooling parameter is changed to 0.997, iteration is reduced to 1304. By changing random flip k from 10 to 6 and cooling parameter $\alpha$ from 0.998 to 0.997, the numbers of random flips and iteration are reduced by 40% and 33%, respectively.

Zoom In Zoom Out Reset image size

4.3. EI Iteration

In IoT device applications, ReRAM CiM deals with inference read operation. ReRAM causes bit-error at high read cycles due to read disturb. ²⁹⁾ The tolerance of read disturb is assessed as iteration can be regarded as read cycle. Figure 11 shows success probability with the different error injection timings of 11(a) 10 bit, 11(b) 7 bit, and 11(c) 4 bit precision. As EI iteration (EI iteration) is delayed, in 4 bit precision, success probability increases although in both 10 bit and 7 bit precision the success probability does not change drastically. As shown in Fig. 11(c), as EI iteration is delayed from 196th to 587th, the acceptable BER increases from 8% to 10%. Figure 12 shows that the SA results in 4 bit precision with 12(a) and 12(c) 196th EI iteration and with 12(b) and 12(d) 587th EI iteration. BER of 10% is injected. Figures 12(a) and 12(b) show temperature and energy with iteration. Figures 12(c) and 12(d) show the obtained value and weight, which are the total value and total weight of the items selected to be packed in the knapsack, with iteration. In the 196th EI iteration, as temperature decreases, energy also decreases as shown in Fig. 12(a). However, the energy is not a minimum value. Therefore, the total value and weight are obtained as the near-optimal answer as shown in Fig. 12(c). On the other hand, in the 587th EI iteration, energy decreases to a minimum value with decreasing temperature as shown in Fig. 12(b), and the obtained total value and weight become maximum as shown in Fig. 12(d). In other words, a 20% delay of EI iteration increases the acceptable BER for read disturb by 25%.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

4.4. Hyperparameter optimization

One of the difficulties of combinatorial optimization problems is to determine complex parameters. ³⁰⁾ In the knapsack problem, hyperparameters are responsible for the requirement that the total weight of the selected items must be under the knapsack capacity. While the conventional Hamiltonian has 3 hyperparameters, $\sigma ,$ $\lambda ,$ and $\mu$ [Eq. (4)], the proposed Hamiltonian has 2 hyperparameters, $\sigma$ and $\mu ,$ reduced by the introduction of log-encoding [Eq. (3)]. Figure 13 shows success probability with the fixed hyperparameter $\mu =1\,or\,2$ and the different hyperparameter $\sigma .$ When hyperparameter $\sigma$ is 1, success probability is low regardless of hyperparameter $\mu .$ If hyperparameter $\sigma$ is too small, it is likely to select bigger items and less likely to select smaller items with higher values. As shown in Fig. 13(a), as for $\mu =1,$ success probability in 10 bit precision is highest where $\sigma =50$ while success probability in 6 bit precision where $\sigma =20.$ As shown in Fig. 13(b), as for $\mu =2,$ success probability in 10 bit precision is worse while success probability in 9 bit precision with $\sigma =40$ is the best, compared with $\mu =1.$ Figure 14 shows success probability with error injection with the different hyperparameter $\sigma .$ As shown in Fig. 14(a), in 10 bit precision with error injection, success probability is not necessarily the best when $\sigma =50.$ Figure 14(b) shows the success probability in 6 bit precision with error injection. Compared with 10 bit precision, even when BER is the same percentage, the optimal hyperparameters are different as bit precision changes. Hyperparameter optimization $\sigma$ is applied in accordance with BER and bit precision because hyperparameters influence success probability as mentioned above. Figure 15 shows the comparison of success probability between the fixed and the optimized hyperparameters with BER. As shown in Fig. 15(a), hyperparameter optimization improves success probability by 1%, compared with the fixed hyperparameter in BER of 3% and 7% in 10 bit precision. In 6 bit precision as shown in Fig. 15(b), hyperparameter optimization improves success probability by up to 1% in BER from 0% to 5%. Figure 16 shows the comparison of success probability between the fixed and the optimized hyperparameters with bit precision. Without error shown in Fig. 16(a), success probability with the optimized hyperparameter is up to 1% higher than that with the fixed hyperparameter except for 10 bit precision. With BER of 3% shown in Fig. 16(b), success probability with the optimized hyperparameter is also up to 1% higher than that with the fixed hyperparameter except 7 bit and 8 bit precision. Hyperparameter optimization improves success probability by up to 1% for both BER and bit precision. Hyperparameter optimization can be effective in the case of extremely high accuracy required by applied technology.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

This paper proposes memory error tolerant and tunable ReRAM CiM with 97.6% array area reduction for solving the knapsack problem, one of the combinatorial optimization problems. The proposed ReRAM CiM solves the knapsack problem by log-encoding SA. "0" and "1" error injection to reproduce ReRAM device errors indicates that the asymmetric ReRAM device reliability improves the acceptable BER by 10 times and the acceptable bit precision to 4 bit. As for SA, random flips k, which prevent energy from staying at local minimums, and cooling parameter $\alpha$, which determines the iteration of SA, are adjusted. Thus, the numbers of random flips and the iteration are reduced by 40% and 33%, respectively. In order to evaluate the read disturb of the ReRAM device, the EI iteration for the inference operation of SA is changed. Consequently, the delay of error injection timing increases the acceptable BER for read disturb by 25%. Furthermore, by optimizing hyperparameters of Hamiltonian in accordance with BER and bit precision, success probability improves by 1%. Hyperparameter optimization can be effective for applied technology that requires extremely high accuracy.

Table I.  _{Summary of this work. Acceptable ≥90% success probability. (a) Array area of ReRAM CiM (Fig. 3). (b) Error injection. (c) Error injection with random flip k and cooling parameter α (Fig. 10). (d) Error injected iteration [Fig. 11(c)]. (e) Success probability with hyperparameter optimization (Figs. 15 and 16).}.

The authors thank R. Yasuharssa and T. Mikawa of Nuvoton Technology Corporation Japan for their ReRAM support. This work is based on results obtained from a project commissioned by the New Energy and Industrial Technology Development Organization (NEDO).