Multistability of competitive neural networks with time-varying and distributed delays

Abstract

In this paper, with two classes of general activation functions, we investigate the multistability of competitive neural networks with time-varying and distributed delays. By formulating parameter conditions and using inequality technique, several novel delay-independent sufficient conditions ensuring the existence of $3^N$ equilibria and exponential stability of $2^N$ equilibria are derived. In addition, estimations of positively invariant sets and basins of attraction for these stable equilibria are obtained. Two examples are given to show the effectiveness of our theory.

Introduction

It is well known that neural networks play important roles in many applications, such as classification, associative memory, image processing, pattern recognition, parallel computation, optimization problem, and decision making etc [1], [2], [3], [4]. The theory on the dynamics of the networks has been developed according to the purposes of the applications. On the one hand, in the applications to parallel computation and optimization problem, the existence of a computable solution for all possible initial states is the best situation. Mathematically, this means that an equilibrium of the networks exists and any state in the neighborhood converges to the equilibrium, which is called “monostability” of networks. On the other hand, existence of many equilibria is a necessary feature in the applications of neural networks to associative memory storage, pattern recognition, and decision making. The notion of “multistability” of networks describes coexistence of multiple stable patterns such as equilibria or periodic orbits.

Competitive neural networks (CNNs) with different time scales are proposed in [5], which model the dynamics of cortical cognitive maps with unsupervised synaptic modifications. In this model, there are two types of state variables, that of the short-term memory (STM) describing the fast neural activity and that of the long-term memory (LTM) describing the slow unsupervised synaptic modifications. A typical form for multitime scale CNNs with time-varying and distributed delays is described by the following functional differential equations

$$\{\begin{array}{c}\text{<mi mathvariant="bold" is="true">STM</mi>}:ϵ\frac{d{x}_i\left(t\right)}{dt}=-a_i{x}_i\left(t\right)+\sum _{j=1}^N{D}_{ij}{f}_j\left(x_j\left(t\right)\right)+\sum _{j=1}^N{D}_{ij}^{\tau }{f}_j\left(x_j(t-{\tau }_{ij}\left(t\right))\right)+\sum _{j=1}^N{\overline{D}}_{ij}{\int }_{-\infty }^t{K}_{ij}(t-s)f_j\left(x_j\left(s\right)\right)ds+B_i\sum _{j=1}^P{m}_{ij}\left(t\right)y_j+I_i\text{,}\\ \text{<mi mathvariant="bold" is="true">LTM</mi>}:\frac{d{m}_{ij}\left(t\right)}{dt}=-m_{ij}\left(t\right)+y_j{f}_i\left(x_i\left(t\right)\right)\text{,}i=1,2,\dots ,N,j=1,2,\dots ,P\text{,}\end{array}$$

where $x_i\left(t\right)$ is the neuron current activity level, $a_i>0$ is the time constant of the neuron, $f_j\left(x_j\left(t\right)\right)$ is the output of neurons, $m_{ij}\left(t\right)$ is the synaptic efficiency, $y_j$ is the constant external stimulus, $D_{ij}$ represents the connection weight between the $i$th neuron and the $j$th neuron, $B_i$ is the strength of the external stimulus, $ϵ$ is the time scale of STM state, $D_{ij}^{\tau }$ and ${\overline{D}}_{ij}$ represent the synaptic weight of delayed feedback, $I_i$ is the constant input, ${\tau }_{ij}\left(t\right)$ corresponds to the transmission delay and satisfies $0<{\tau }_{ij}\left(t\right)<{\tau }_{ij}$ (${\tau }_{ij}$ is a positive constant).

After setting $S_i\left(t\right)={\sum }_{j=1}^P{m}_{ij}\left(t\right)y_j=y^T{m}_i\left(t\right)$, where $y={(y_1,y_2,\dots ,y_P)}^T$, $m_i\left(t\right)={(m_{i1}\left(t\right),m_{i2}\left(t\right),\dots ,m_{iP}\left(t\right))}^T$, and summing up the LTM over $j$, the networks (1) can be rewritten in the following form

$$\{\begin{array}{c}\text{<mi mathvariant="bold" is="true">STM</mi>}:ϵ\frac{d{x}_i\left(t\right)}{dt}=-a_i{x}_i\left(t\right)+\sum _{j=1}^N{D}_{ij}{f}_j\left(x_j\left(t\right)\right)+\sum _{j=1}^N{D}_{ij}^{\tau }{f}_j\left(x_j(t-{\tau }_{ij}\left(t\right))\right)+\sum _{j=1}^N{\overline{D}}_{ij}{\int }_{-\infty }^t{K}_{ij}(t-s)f_j\left(x_j\left(s\right)\right)ds+B_i{S}_i\left(t\right)+I_i\text{,}\\ \text{<mi mathvariant="bold" is="true">LTM</mi>}:\frac{d{S}_i\left(t\right)}{dt}=-S_i\left(t\right)+{\left|y\right|}^2{f}_i\left(x_i\left(t\right)\right)\text{,}i=1,2,\dots ,N\text{,}\end{array}$$

where ${\left|y\right|}^2=y_1^2+\cdots +y_P^2$ is a constant, without loss of generality, the input stimulus $y$ is assumed to be normalized with unit magnitude ${\left|y\right|}^2=1$, and the fast time-scale parameter $ϵ$ is also assumed to be unit, then the above networks are simplified as

$$\{\begin{array}{c}\text{<mi mathvariant="bold" is="true">STM</mi>}:\frac{d{x}_i\left(t\right)}{dt}=-a_i{x}_i\left(t\right)+\sum _{j=1}^N{D}_{ij}{f}_j\left(x_j\left(t\right)\right)+\sum _{j=1}^N{D}_{ij}^{\tau }{f}_j\left(x_j(t-{\tau }_{ij}\left(t\right))\right)+\sum _{j=1}^N{\overline{D}}_{ij}{\int }_{-\infty }^t{K}_{ij}(t-s)f_j\left(x_j\left(s\right)\right)ds+B_i{S}_i\left(t\right)+I_i\text{,}\\ \text{<mi mathvariant="bold" is="true">LTM</mi>}:\frac{d{S}_i\left(t\right)}{dt}=-S_i\left(t\right)+f_i\left(x_i\left(t\right)\right)\text{,}i=1,2,\dots ,N\text{,}\end{array}$$

where the delay kernels $K_{ij}\left(s\right):[0,+\infty )\to [0,+\infty )$ are piecewise continuous integral functions, and they satisfy

$${\int }_0^{+\infty }{K}_{ij}\left(s\right)ds=1\text{,}{\int }_0^{+\infty }{K}_{ij}\left(s\right)e^{\mu s}ds<+\infty$$

for some positive constant $\mu$ and $i,j=1,2,\dots ,N$.

In the past few years, the monostability analysis of neural networks with time-varying and/or distributed delays has been developed [6], [7], [8], [9], [10], [11], [12], [13]. In particular, the theory of unique equilibrium point and global convergence to the equilibrium point for CNNs with their various generalizations has been extensively studied, see [5], [14], [15], [16], [17]. Recently, the multistability analysis of neural networks has attracted the attention of many researchers [18], [19], [20], [21]. In [18], with unsaturated piecewise linear activation function $f\left(x\right)=max\{0,x\}$, the multistability of system (3) without delay, that is $D_{ij}^{\tau }={\overline{D}}_{ij}=0(i,j=1,2,\dots ,N)$, was investigated by using local inhibition and constructing an energy-like function. The result therein strongly relies on the piecewise linearity and unsaturation of activation functions. In [19], [20], the multistability of neural networks without and with constant delays was studied by formulating parameter conditions. Inspired by [19], [20], in this paper, we shall study the multistability of system (3) with two classes of general activation functions. Firstly, we derive conditions ensuring the existence of $3^N$ equilibria for system (3) with two classes of general activation functions, through constructing parameter conditions by a geometrical observation. Secondly, we establish a series of new criteria on the exponential stability of $2^N$ equilibria for the networks above by means of inequality technique. In addition, estimations of positively invariant sets and basins of attraction for these stable stationary solutions are derived.

This paper is organized as follows. In Section 2, we consider two classes of activation functions which are commonly employed in neural networks. We then obtain conditions for the existence of $3^N$ equilibria. In Section 3, we show that, with additional conditions, there are $2^N$ regions in ${\mathbb{R}}^{2N}$, which are positively invariant under the flow generated by system (3). Subsequently, it is argued that these $2^N$ equilibria are exponentially stable. Two numerical simulations are given to illustrate our theory and distinct dynamical behaviors for different activation functions in Section 4. Finally, concluding remarks are summarized in Section 5.