Optimization of dynamic neural fields

doi:10.1016/S0925-2312(00)00328-3

Neurocomputing

Volume 36, Issues 1–4, February 2001, Pages 225-233

https://doi.org/10.1016/S0925-2312(00)00328-3 Get rights and content

Abstract

There is a growing interest in using dynamic neural fields for modeling biological and technical systems, but constructive ways to set up such models are still missing. We discuss gradient-based, evolutionary and hybrid algorithms for data-driven adaptation of neural field parameters. The proposed methods are evaluated using artificial and neuro-physiological data.

Introduction

Neural fields (NFs) have been introduced as mathematical descriptions of cortical neural tissues, where information processing takes place in the form of excitation patterns [22], [1], [12], [7]. They permit a systematic treatment of dynamical processes not only in distributed neural representations, but also in the context of the dynamical system approach to perception and behavior, behavior-based robotics, computer vision and sensor fusion, see [17], [8], [19] for further references. However, up to now there exists no constructive way to set up NF models. In order to clarify and substantiate experimental findings, an expert has to adjust the parameters of the NF. Since the fields reveal a highly non-linear dynamic behavior, this is a non-trivial task even for qualitative modeling. The choice of an appropriate set of parameters is an important problem also for technical applications, where NFs have to be designed to perform a given task, e.g., the control of robot behavior [4], [5].

The most common techniques for data-driven parameter adaptation of recurrent neural networks (RNNs) are gradient-based methods [6], [14]. In this study, we adopt these algorithms for adjusting the parameters of NFs. For comparison, we apply an elaborated evolution strategy [9], [10] to NF optimization. We present a hybrid algorithm that combines the local search of gradient-based methods with the more volume-oriented search of evolutionary algorithms.

Section snippets

Neural field model

We restrict our investigation to a one-dimensional NF formulation based on the model first analyzed by Amari [1]. However, the extension of the optimization algorithms to higher dimensional fields and coupled systems of NFs is straightforward.

Let u_x(t) describe the average membrane potential at time t of a neuron located at site x of the field. Its average activity (mean firing rate) is given by the transfer function $f [u_{x} (t)]$ , which is assumed to be of sigmoidal shape, $f (u)=α/(1+ exp [β(u−u_{shift}$

Gradient-based optimization

The goal is to adapt the NF parameters such that starting from the relaxed field for all t∈[0,T] and x=1,…,n either the potential u_x(t) or the average activity $f [u_{x} (t)]$ follows a desired trajectory d_x(t) for a given stimulus pattern s_x(t). Therefore, we define an error function $E= ∫ 0 T E(t) d t$ and perform gradient descent on E(t) with respect to the parameters $θ$ . For each parameter $θ_{i}, i=1,…,m$ , we have to calculate $d E d θ_{i} = ∫ t=0 T ∑ x=1 n e_{x} (t) d u_{x} (t) d θ_{i} d t with e_{x} (t)=μ_{x} (t) ∂E(t) ∂u_{x} (t),$ where the masking function μ

Evolution strategy

Evolutionary algorithms (EAs) are a class of direct, easy to parallelize optimization methods inspired by natural evolution and have been applied successfully in the field of neural networks [23]. A variety of EAs for parameter optimization exists [3], in our study a state-of-the-art evolution strategy (ES) is employed, namely the $(μ/μ_{I}, λ)$ -CMA-ES [9], [10]. The notation $(μ/μ_{I}, λ)$ indicates that the new parent population consists of the μ best of the λ offspring and the use of intermediate

Artificial data

The first task is to reproduce a synthetic potential pattern shown in Fig. 1. It is generated by an NF as described by Eq. (1). The model consists of 100 neurons and is iterated for 100 time steps. The parameters are g_ex=40, σ_ex=10, g_inh=60, σ_inh=40, h=3, α=1.2, β=−1.5, u_shift=0.2, and τ=5. Two Gaussian stimuli $(g_{s} =20, σ_{s} =10)$ are given for the first 50 time steps at the positions 35 and 65, respectively; the activation fuses to a single peak. We assume that stimulus shape, position and duration

Discussion

Gradient-based techniques can be adopted for NF optimization, where forward methods should be preferred. Our experiments show that the optimization of NFs is a difficult task; the algorithms tend to get trapped in local minima. In particular, bifurcations in the NF dynamics exacerbate the search process. These difficulties can partly be overcome by incorporating additional expert knowledge, e.g., by ensuring that necessary conditions for a desired type of solution are met. Nonetheless, in our

Acknowledgements

This work was supported by the Ministerium für Schule und Weiterbildung, Wissenschaft und Forschung des Landes Nordrhein-Westfalen under Grant 516 - 102 001 98.

References (23)

K. Arai et al.
A distributed model of the saccade system: simulations of temporally perturbed saccades using position and velocity feedback
Neural Networks
(1999)
T. Bergener et al.
Complex behaviour by means of dynamical systems for an anthropomorphic robot
Neural Networks
(1999)
W. Erlhagen et al.
The distribution of neural population activation as a tool to study interaction and integration in cortical representations
J. Neurosci. Methods
(1999)
G. Schöner et al.
The dynamic neural field theory of motor programming: arm and eye movements
S. Amari
Dynamics of pattern formation in lateral-inhibition type neural fields
Biol. Cybern.
(1977)
T. Bäck et al.
An overview of evolutionary algorithms for parameter optimization
Evol. Comput.
(1993)
E. Bicho et al.
Target representation on an autonomous vehicle with low-level sensors
Internat. J. Robotics Res.
(2000)
K. Doya
Recurrent networks: supervised learning
M.A. Giese
Dynamic Neural Field Theory for Motion Perception
(1999)
N. Hansen, A. Ostermeier, Convergence properties of evolution strategies with the derandomized covariance matrix...

N. Hansen, A. Ostermeier, Completely derandomized self-adaptation in evolution strategies, Evol. Comput. (special issue...

Cited by (23)

Data-driven parameter estimation in stochastic dynamic neural fields by state-space approach and continuous-discrete extended Kalman filtering
2023, Digital Signal Processing: A Review Journal
This work examines the state and parameter estimation problems for Stochastic Dynamic Neural Field (SDNF) models. Unlike other studies, we suggest a general approach based on the Galerkin method that reduces the SDNF model to a standard state-space representation. This naturally yields nonlinear continuous-discrete stochastic systems with complicated nonlinear dynamics and linear measurements of a high dimension because of a spatial approximation. This in turn paves a systematic way for a strong integration of nonlinear Bayesian filtering methods into mathematical neuroscience. Because of a high resolution utilized to keep an accurate spatial approximation, a key aspect of nonlinear filters to be derived for the SDNF models is their computational budgets. In this paper, we derive the sequential Itô-Taylor-based continuous-discrete Extended Kalman filter for state and parameter estimation of the SDNF models. The novel method processes each measurement vector in a component-like manner and, hence, avoids an inversion of a large matrix. Thus, the newly-derived sequential estimator is faster and, additionally, more numerically stable compared to the previously developed SDNF-oriented estimators. Besides, the novel filter is an adaptive method, i.e. it solves both problems, which are the state and parameter estimation, in parallel. Finally, the proposed method is applied to numerical examples to provide the comparative study and to show the effectiveness of our result.
Sequential method for fast neural population activity reconstruction in the cortex from incomplete noisy measurements
2022, Computers in Biology and Medicine
Citation Excerpt :
Our novel technique relies upon a Galerkin-type spectral approximation utilized within the conventional state-space approach. We pay attention to the fact that translating a stochastic system into its state-space form, which is our special goal in the present research, creates a straightforward and fruitful way to the data-driven adaptation and parameter estimation [6], filtering, prediction [7] and smoothing. In addition, we remark that some nonlinear Bayesian filtering methods and, in particular, the unscented Kalman filter (UKF) have been already utilized for parameter estimations in dynamic NFs in engineering literature, e.g., in Ref. [8].
During recent years there has been a growing interest in stochastic dynamic neural fields employed for modeling and predictions in biomedical and technical systems. In this paper, given some incomplete noisy data available from sensors, we propose and explore a state estimation method for fast restorations of membrane potential in the cortex based on such measurements and the Amari equation used for simulations of neural population activity in a stochastic setting. Our novel technique relies upon a Galerkin-type spectral approximation utilized within the conventional state-space approach. Translating a stochastic system into its state-space form creates a straightforward and fruitful way to the data-driven parameter estimation, filtering, prediction and smoothing. The present study is particularly focused on establishing a nonlinear stochastic Galerkin-spectral-approximation-induced system of large size, which is further estimated by the traditional extended Kalman filter (EKF). The efficiency of calculations is the main purpose of our research. That is why the fast filtering solution devised is based on processing the incoming data incrementally, that is, by processing measurements one at a time, rather than handling them as a unified high-dimensional vector. Such sequential filters suit well for dealing with large data sets as well as with real-time on-line computations. Also, their derivation and substantiation is of great interest in the context of neural network training because of large stochastic systems arisen there. In comparison to the batch filtering, our novel algorithm reduces the computational cost of membrane potential reconstructions in terms of the amount of grid nodes N accepted in the underlying spacial discretization, significantly. Apart from its computation efficiency, this sequential method is more robust to round-off errors committed within a computer-based finite precision arithmetics than the classical EKF because of the (N × N)-matrix inversion elimination from such membrane potential calculations. The superior performance of our technique is examined and confirmed in comparison to the batch one on two known scenarios in the dynamic neural field modeling.
Neuroevolution strategies for episodic reinforcement learning
2009, Journal of Algorithms
Because of their convincing performance, there is a growing interest in using evolutionary algorithms for reinforcement learning. We propose learning of neural network policies by the covariance matrix adaptation evolution strategy (CMA-ES), a randomized variable-metric search algorithm for continuous optimization. We argue that this approach, which we refer to as CMA Neuroevolution Strategy (CMA-NeuroES), is ideally suited for reinforcement learning, in particular because it is based on ranking policies (and therefore robust against noise), efficiently detects correlations between parameters, and infers a search direction from scalar reinforcement signals. We evaluate the CMA-NeuroES on five different (Markovian and non-Markovian) variants of the common pole balancing problem. The results are compared to those described in a recent study covering several RL algorithms, and the CMA-NeuroES shows the overall best performance.
Learning recurrent neural models with minimal complexity from neural tuning data
2003, Neurocomputing
A learning algorithm for the estimation of the structure of nonlinear recurrent neural models from neural tuning data is presented. The proposed method combines support vector regression with additional constraints that result from a stability analysis of the dynamics of the fitted network model. The optimal solution can be determined from a single convex optimization problem that can be solved with semidefinite programming techniques. The method successfully estimates the feed-forward and the recurrent connectivity structure of neural field models using as data only examples of stable stationary solutions of the neural dynamics. The class of neural models that can be learned is quite general. The only a priori assumptions are the translation invariance and the smoothness of the feed-forward and recurrent spatial connectivity profile. The efficiency of the method is illustrated by comparing it with estimates based on radial basis function networks and support vector regression.
Evolving field models for inhibition effects in early vision
2002, Neurocomputing
We present a nonlinear field model based on linear couplings for inhibition effects in early vision. The model is fitted to data from single unit recordings in the primary visual cortex of the cat. We focus on the prominent effect that responses to second stimuli are amplified, reduced, or unaffected depending on the temporal and spatial distance of the stimuli. The model is adjusted using an elaborated self-adaptive evolution strategy resulting in an accurate, easy to interpret, and well generalizing model.
A neurocomputational theory of action regulation predicts motor behavior in neurotypical individuals and patients with Parkinson's disease
2022, PLoS Computational Biology
Surviving in an uncertain environment requires not only the ability to select the best action, but also the flexibility to withhold inappropriate actions when the environmental conditions change. Although selecting and withholding actions have been extensively studied in both human and animals, there is still lack of consensus on the mechanism underlying these action regulation functions, and more importantly, how they inter-relate. A critical gap impeding progress is the lack of a computational theory that will integrate the mechanisms of action regulation into a unified framework. The current study aims to advance our understanding by developing a neurodynamical computational theory that models the mechanism of action regulation that involves suppressing responses, and predicts how disruption of this mechanism can lead to motor deficits in Parkinson’s disease (PD) patients. We tested the model predictions in neurotypical individuals and PD patients in three behavioral tasks that involve free action selection between two opposed directions, action selection in the presence of conflicting information and abandoning an ongoing action when a stop signal is presented. Our results and theory suggest an integrated mechanism of action regulation that affects both action initiation and inhibition. When this mechanism is disrupted, motor behavior is affected, leading to longer reaction times and higher error rates in action inhibition.

View all citing articles on Scopus

View full text

LettersOptimization of dynamic neural fields

Abstract

Introduction

Section snippets

Neural field model

Gradient-based optimization

Evolution strategy

Artificial data

Discussion

Acknowledgements

Neural Networks

Neural Networks

J. Neurosci. Methods

Dynamics of pattern formation in lateral-inhibition type neural fields

Biol. Cybern.

An overview of evolutionary algorithms for parameter optimization

Evol. Comput.

Target representation on an autonomous vehicle with low-level sensors

Internat. J. Robotics Res.

Recurrent networks: supervised learning

Dynamic Neural Field Theory for Motion Perception

Letters
Optimization of dynamic neural fields