Statistical mechanics is applied to study the generalization properties of a two-layer neural network trained to implement a linearly separable problem. For a stochastic learing algorithm the generalization error as a function of the training set size is calculated exactly. The network with three hidden units experiences two first-order phase transitions due to an asymmetric freezing of the hidden units. Compared to a simple perceptron the committee machine is found to generalize worse.

• Received 26 August 1992

DOI:https://doi.org/10.1103/PhysRevA.46.R6185