The formation of ensemble of artificial neural networks has attracted attentions of researchers in the machine learning and statistical inference domains. It has been shown that combining different neural networks could improve the generalization ability of the learning machine. One challenge is when to stop the training or evolution of the neural networks to avoid overfitting. In this paper, we show that different early stopping criteria based on (i) the minimum validation fitness of the ensemble, and (ii) the minimum of the average population validation fitness could generalize better than the survival population in the last generation. The proposition was tested on four different ensemble methods: (i) a simple ensemble method, where each individual of the population (created and maintained by the evolutionary process) is used as a committee member, (ii) ensemble with island model as a diversity promotion mechanism, (iii) a recent successful ensemble method namely ensemble with negative correlation learning and (iv) an ensemble formed by applying multi-objective optimization. The experimental results suggested that using minimum validation fitness of the ensemble as an early stopping criterion is beneficial.