ScienceDirect® Home Skip Main Navigation Links
You have guest access to ScienceDirect. Find out more.
 
Home
Browse
My Settings
Alerts
Help
 Quick Search
 Search tips (Opens new window)
    Clear all fields    
Neural Networks
Volume 13, Issue 3, April 2000, Pages 365-375
 
Font Size: Decrease Font Size  Increase Font Size
 Abstract - selected
Article
Purchase PDF (162 K)

Article Toolbox
 
 
 
Related Articles in ScienceDirect
View More Related Articles
 
View Record in Scopus
 
doi:10.1016/S0893-6080(00)00015-0    
How to Cite or Link Using DOI (Opens New Window)

Copyright © 2000 Elsevier Science Ltd. All rights reserved.

Contributed article

Information complexity of neural networks

Purchase the full-text article



References and further reading may be available for this article. To view references and further reading you must purchase this article.

M. A. KonCorresponding Author Contact Information, E-mail The Corresponding Author, a and L. Plaskotab

a Department of Mathematics, Boston University, Boston, MA 02215, USA

b Department of Mathematics, Informatics, and Mechanics, Warsaw University, 2 Banacha Street, 02-097 Warsaw, Poland


Received 9 February 1999;
accepted 25 January 2000.
Available online 19 May 2000.

Abstract

This paper studies the question of lower bounds on the number of neurons and examples necessary to program a given task into feedforward neural networks. We introduce the notion of information complexity of a network to complement that of neural complexity. Neural complexity deals with lower bounds for neural resources (numbers of neurons) needed by a network to perform a given task within a given tolerance. Information complexity measures lower bounds for the information (i.e. number of examples) needed about the desired input–output function. We study the interaction of the two complexities, and so lower bounds for the complexity of building and then programming feed-forward nets for given tasks. We show something unexpected a priori—the interaction of the two can be simply bounded, so that they can be studied essentially independently. We construct radial basis function (RBF) algorithms of order n3 that are information-optimal, and give example applications.

Author Keywords: Feedforward neural network; Complexity; Information complexity; Neural complexity; Radial basis functions; RBF networks; Learning

Article Outline

1. Introduction
2. Definitions and main result
3. Examples
4. Neural complexity (k=∞)
Information complexity (n=∞)Question 2.  Given an unknown function f in some class, what is the smallest number k of examples Image for which it is (theoretically) possible to estimate f within error ε (assuming unlimited access to hidden units)?
Interaction of information and neural complexitiesQuestion 3.  Given information about f with k examples, what is the best network approximating f which uses at most n neurons in the hidden layer?
Acknowledgements
References


Corresponding Author Contact Information Corresponding author. Tel.: +1-617-353-2560; fax: +1-617-353-8100; email: mkon@bu.edu


Neural Networks
Volume 13, Issue 3, April 2000, Pages 365-375
 
Home
Browse
My Settings
Alerts
Help
Elsevier.com (Opens new window)
About ScienceDirect  |  Contact Us  |  Information for Advertisers  |  Terms & Conditions  |  Privacy Policy
Copyright © 2009 Elsevier B.V. All rights reserved. ScienceDirect® is a registered trademark of Elsevier B.V.