Abstract

The comparative performances of distributed and local neural networks for the speech recognition problem are investigated. We consider a feed-forward network with one or more hidden layers. Depending on the response characteristics of the hidden units, we name the network distributed or local. If the hidden units use the sigmoid non-linearity, then hidden units have a global response and we call such networks distributed. If each hidden unit responds only to inputs in a certain local region in the input space, then the network is local. Neighbor and prototype based approaches are of this type. As examples of the distributed approach with sigmoidal units, we employ the back-propagation rule with three error measures: mean square error, cross entropy, and combinational performance. As for the local methods, we use k-nearest neighbor, learning vector quantization, grow and learn, and Gaussian-based weighted approximation methods. Phoneme recognition experiments are conducted using the /b, d, g, m, n, N/ set of the Japanese vocabulary for the speaker dependent case. Three criteria are taken for comparison: correct classification of the test set, network size, and learning time. We found that distributed networks generalize better than local networks but require longer training and more precise computation. Local networks learn very quickly, but do not generalize well and use more memory.

Author Keywords: Speech recognition; Artificial neural networks; Distributed neural network; Back-propagation; Local networks; k-nearest neighbor rule; Grow and learn method; Learning vector quantization

Article Outline

• References