Abstract

In a typical optimization problem, the main goal is to search for the appropriate values of the variables that provide the optimal solution of the given function. In artificial neural networks (ANN), this translates to the minimization of the error surface during training such that misclassification is minimized during generalization. However, since optimal training performance does not necessarily imply optimal generalization due to the possibility of overfitting or underfitting, we developed SEPA (Structure Evolution and Parameter Adaptation) which addressed these issues by simultaneously evolving ANN structure and weights. Since SEPA primarily relies on the perturbation function to bring variation in its population, this follow-up study aims to find out SEPAs evolvability, optimality, and robustness in other perturbation functions. Our findings indicate that SEPAs optimal generalization performances are stable and robust from the effect of the different perturbation functions. This is due to the feedback loop between its architecture evolution and weight adaptation such that any shortcoming of the former is compensated by the latter, and vice versa. Our results strongly suggest that proper ANN design requires simultaneous adaptation of ANN structure and weights to avoid one-sided or bias convergence to either the weight or architecture space.

Keywords: Evolutionary neural network; Stochastic adaptation; Optimization; Classification; Perturbation function; Evolvability

Article Outline

1. Introduction

2. SEPA

3. Experiments and results

3.1. Glass problem

3.2. Diabetes problem

3.3. Heart problem

3.4. Six-bit parity problem

3.5. Boxplots

4. Discussion

4.1. Sensitivity of SEPA to the choice of perturbation function

4.2. Effective mutation strategy

4.3. Stability and optimality

5. Conclusion

Acknowledgements

Appendix A. Algorithms

Appendix B. Supplementary Data

Appendix C. Supplementary Data

References