Contributed articleSingular-continuous nowhere-differentiable attractors in neural systems
Introduction
Dynamical states in neural systems can be represented by various kinds of attractors in dynamical systems: a stationary state by fixed point, a periodically oscillatory state by limit cycle, a quasi-periodic state by torus, a low-dimensional chaotic state by strange attractor, and a high-dimensional itienrant state by itinerant attractor. Among others, recently an itinerant attractor has been highlighted in the research of complex systems like neural systems, where the new notion of chaotic itinerancy was proposed. (Ikeda et al., 1989; Kaneko, 1990; Tsuda, 1991a, Tsuda, 1991b). These states have been widely investigated and also observed in experiments (for epoch-making experimental works including chaotic itinerancy in neural systems, see Freeman, 1987, Freeman, 1994, Freeman, 1995a, Freeman, 1995b; Skarda and Freeman, 1987; Kay et al., 1995). There is still, however, another dynamical attractor which was named `strange nonchaotic attractor' by Grebogi, Otto, Kaplan and Yorke (Kaplan and Yorke, 1979; Kaplan et al., 1984; Grebogi et al., 1984). Here, `strange' means the presence of Cantor sets, and `nonchaotic' the absence of positive Lyapunov exponents.
A possible mechanism of strange nonchaotic attractors was recently proposed by Rössler et al. (1992). Actually, the attractors can be represented by singular-continuous nowhere-differentiable (SCND) functions. We will provide the definition of this class of functions in Section 3, where we will develop a theory to explain `strange' dynamic characters of our model neural network. It should, however, be noted that SCND attractors must be, in general, different from strange nonchaotic attractors since the former appears in contraction subspace of a whole space including chaotic components, whereas the latter does not include chaotic components1.
Any theories, models, and even experimental observations for this attractor in neural systems have not reported so far, except that we showed the presence of such an attractor in neural networks (Tsuda, 1996). Furthermore, the questions have not yet been elucidated if this attractor could be observed in biological neural networks, and also if it could subserve the information processings in brain. In this paper, we present a small scale neural network model which exhibits SCND attractors. Our aim is to elucidate the former question and for the latter to present hypotheses based on the numerical simulations of the model.
A neural network model exhibiting SCND attractors will be presented in Section 2. In Section 3, a general scheme giving rise to SCND attractors will be addressed, whereby the mechanism of SCND attractors obtained in Section 2will be elucidated. In Section 4, various characteristics of the obtained SCND attractors will be studied. Section 5will be devoted to conclusion and discussion, where hypotheses on information processings with SCND attractors and the possibility of the observation of SCND attractors in biological neural networks will be addressed.
Section snippets
A model
Our model neural network presented here is of a small scale. The model consists of only three neurons, two of which are called here a `static' neuron and the other one a `dynamic' neuron. A static neuron's activity eventually relaxes to stationary firings which is represented by a fixed point in phase space, whereas a dynamic neuron fires chaotically, thus it is represented by chaotic attractor. It would be convenient to use a chaotic neuron model introduced by Aihara et al. (1990) in order to
Continuous but nowhere-differentiable functions
Continuous but nowhere-differentiable functions have been widely investigated (Takagi, 1973; Titchmarsh, 1985; Hata, 1988aHata, 1988bHata, 1994). In particular, Yamaguti (1989), and Hata and Yamaguti (1984) elucidated a relation between fractals (Mandelbrot, 1982) and chaos. Recently, perhaps the simplest example of this class of functions has been proposed by Katsuura (1991). Katsuura's function (named by Rössler) is obtained from the graph of asymptotic form of unit square transformed by a
Dynamic features of neuro-SCND attractors
In order to characterize the attractor seen on the cross-section, we calculated the following statistical quantities: invariant measure, Lyapunov spectrum, mutual information, entropies, distribution of recurrence time, and dimension. The quantities except for the Lyapunov spectrum and dimension were calculated with some partition, thus giving coarse-grained ones. We changed partition in several systematic ways, and concluded the presence of invariant characters. We also investigated a noise
Conclusion and discussion
We proposed a neural network model exhibiting singular-continuous nowhere-differentiable (SCND) attractors. This is the first finding of SCND attractors in neural systems. We also elucidated a mechanism of SCND attractors with a help of Rössler's construction of singular-continuous nowhere-differentiable functions. In order to characterize SCND attractors, we calculated invariant measure, Lyapunov dimension in terms of Lyapunov spectrum, entropies, and mutual information. Here, all calculations
Acknowledgements
One of the authors (I.T.) would like to express his special thanks to Otto Rössler, Gerold Baier, Axel Hoff and Hans Diebner for their invaluable discussions. He also thanks Reimara and Michal Rössler for encouragement. This work was partially supported by Grant-in-Aid #08279103 for Scientific Research on Priority Areas on `System Theoretical Understandings for Brain Higher Functions,' the Ministry of Education, Science, Sports, and Culture of Japan.
References (40)
- et al.
Chaotic neural networks
Physics Letters A
(1990) Neural mechanisms underlying destabilization of cortex by sensory input
Physica D
(1994)- et al.
Strange attractors that are not chaotic
Physica
(1984) Clustering, coding, switching, hierarchical ordering, and control in network of chaotic elements
Physica D
(1990)- et al.
Extended information in one-dimensional maps
Physica D
(1987) On a theorem of Anosov
Journal of Differential Equations
(1969)- et al.
Analog computation via neural networks
Theoretical Computer Sciences
(1994) Dynamic link of memories–chaotic memory map in nonequilibrium neural networks
Neural Networks
(1992)Can stochastic renewal of maps be a model for cerebral cortex?
Physica, D
(1994)- et al.
Associative dynamics in a chaotic neural network
Neural Networks
(1997)
Simulation of chaotic EEG patterns with a dynamic model of the olfactory system
Biological Cybernetics
On Weierstrass's non-differentiable function
Comptes Rendue Academie de la Science, Paris
Singularities of the Weierstrass type functions
Journal d'Analyse Mathematique
Correction to singularities of the Weierstrass type functions
Journal d'Analyse Mathematique
Takagi function and its generalization
Japan Journal of Applied Mathematics
Maxwell–Bloch turbulence
Progress of Theoretical Physics Supplement
Statistical mechanics of deterministic chaos
Progress of Theoretical Physics
Cited by (21)
A mathematical model for Cantor coding in the hippocampus
2011, Neural NetworksCitation Excerpt :In early studies of recurrent neural networks (Elman, 1990; Pollack, 1991; Tani, 1996), it was reported that Cantor sets emerge in the internal state space of neural networks that deal with temporal sequences such as language and motor commands. Tsuda and colleagues have proposed a Cantor coding scheme in chaos-driven contracting systems (Tsuda & Yamaguchi, 1998), and discussed the possibility of Cantor coding for the formation of episodic memory in the hippocampus (Tsuda, 2001; Tsuda & Kuroda, 2001). Using a simple hippocampal model, they showed that transitions between so-called quasi-attractor states occur in the CA3 recurrent network, and that the path of the state transitions can be hierarchically encoded on the Cantor set in CA1.
Is there chaos in the brain? II. Experimental evidence and related models
2003, Comptes Rendus - BiologiesCitation Excerpt :Such ‘chaos-driven contracting’ systems possess attractors represented by so-called SCND functions (for singular-continuous but nowhere differentiable). These functions which can be viewed [297,298] as ‘fractal images’ with Cantor sets, related to the formation of “episodic memory and primitive thought processes” [291]. It has been predicted that they will be found in the membrane potential of inhibitory neurons driven by chaotic afferents [297].
Errorless description with two rules of cellular automata for digital sound data
2013, International Journal of Bifurcation and ChaosDynamical properties of three-rules set realizing compressive and errorless description of digital sound
2021, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)Synthesis of Weierstrass Fractional Noise for Validation of Polyscale Algorithms
2019, 2019 IEEE Canadian Conference of Electrical and Computer Engineering, CCECE 2019Iterated function systems in the hippocampal CA1
2009, Cognitive Neurodynamics