Abstract

Simple recurrent networks (Elman networks) have been widely used in temporal processing applications. In this study we investigate temporal generalization of simple recurrent networks, drawing comparisons between network capabilities and human performance. Elman networks are trained to generate temporal trajectories sampled at different rates. The networks are then tested with trajectories at the trained rates and other sampling rates, including trajectories representing mixtures of different sampling rates. It is found that for simple trajectories the networks show interval invariance, but not rate invariance. However, for complex trajectories which require greater contextural information, these networks do not seem to show any temporal generalization. Similar results are also obtained using measured speech data. These results suggest that this class of recurrent networks exhibits severe limitations in temporal generalization. Discussions are provided regarding rate invariance and possible ways to achieve it in neural networks. Copyright © 1996 Elsevier Science Ltd

Author Keywords: Neural networks, Temporal generalization, Recurrent networks, Elman networks, Speech processing, Rate, invariance, Interval invariance, Simple sequence, Complex sequence

^*Requests for reprints should be sent to DeLiang Wang, Department of Computer and Information Science, The Ohio State University, 2015 Neil Avenue, Columbus, OH 43210-1277, USA.