Summary
This chapter explores the issue of overfitting in grammar-based Genetic Programming. Tools such as Genetic Programming are well suited to problems in finance where we seek to learn or induce a model from data. Models that overfit the data upon which they are trained prevent model generalisation, which is an important goal of learning algorithms.
Early stopping is a technique that is frequently used to counteract overfitting, but this technique often fails to identify the optimal point at which to stop training. In this chapter, we implement four classes of stopping criteria, which attempt to stop training when the generalisation of the evolved model is maximised.
We show promising results using, in particular, one novel class of criteria, which measured the correlation between the training and validation fitness at each generation. These criteria determined whether or not to stop training depending on the measurement of this correlation - they had a high probability of being the best among a suite of potential criteria to be used during a run. This meant that they often found the lowest validation set error for the entire run faster than other criteria.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agapitos, A., O’Neill, M., Brabazon, A.: Evolutionary Learning of Technical Trading Rules without Data-Mining Bias. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 294–303. Springer, Heidelberg (2010)
Banzhaf, W., Nordin, P., Keller, R., Francone, F.: Genetic programming: An introduction. Morgan Kaufmann, San Francisco (1999)
Becker, L.A., Seshadri, M.: Comprehensibility and overfitting avoidance in genetic programming for technical trading rules. Worcester Polytechnic Institute, Computer Science Technical Report (2003)
Brabazon, A., Dang, J., Dempsey, I., O’Neill, M., Edelman, D.: Natural computing in finance: a review. In: Handbook of Natural Computing: Theory, Springer, Heidelberg (2010)
Brabazon, A., O’Neill, M.: Biologically inspired algorithms for financial modelling. Springer, Heidelberg (2006)
Costelloe, D., Ryan, C.: On improving generalisation in genetic programming. In: Vanneschi, L., Gustafson, S., Moraglio, A., De Falco, I., Ebner, M. (eds.) EuroGP 2009. LNCS, vol. 5481, pp. 61–72. Springer, Heidelberg (2009)
Dempsey, I., O’Neill, M., Brabazon, A.: Foundations in Grammatical Evolution for Dynamic Environments. Springer, Heidelberg (2009)
Domingos, P.: The role of Occam’s razor in knowledge discovery. Data Mining and Knowledge Discovery 3(4), 409–425 (1999)
Gagné, C., Schoenauer, M., Parizeau, M., Tomassini, M.: Genetic programming, validation sets, and parsimony pressure. In: Collet, P., Tomassini, M., Ebner, M., Gustafson, S., Ekárt, A. (eds.) EuroGP 2006. LNCS, vol. 3905, pp. 109–120. Springer, Heidelberg (2006)
Gencay, R., Qi, M.: Pricing and hedging derivative securities with neural networks: Bayesian regularization, early stopping, and bagging. IEEE Transactions on Neural Networks 12(4), 726–734 (2002)
Kushchu, I.: Genetic programming and evolutionary generalization. IEEE Transactions on Evolutionary Computation 6(5), 431–442 (2002)
Luke, S., Panait, L.: Lexicographic parsimony pressure. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 829–836. Morgan Kaufmann, San Francisco (2002)
Luke, S., Panait, L.: A comparison of bloat control methods for genetic programming. IEEE Transactions on Evolutionary Computation 14(3), 309–344 (2006)
Mckay, R.I., Hoai, N.X., Whigham, P.A., Shan, Y., O’Neill, M.: Grammar-based Genetic Programming: a survey. Genetic Programming and Evolvable Machines 11(3-4), 365–396 (2010)
Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)
O’Neill, M., Hemberg, E., Gilligan, C., Bartley, E., McDermott, J., Brabazon, A.: GEVA: grammatical evolution in Java. ACM SIGEVOlution 3(2), 17–22 (2008)
O’Neill, M., Ryan, C.: Grammatical Evolution: Evolutionary automatic programming in an arbitrary language. Kluwer, Dordrecht (2003)
O’Neill, M., Vanneschi, L., Gustafson, S., Banzhaf, W.: Open issues in genetic programming. Genetic Programming and Evolvable Machines 11(3-4), 339–363 (2010)
Paris, G., Robilliard, D., Fonlupt, C.: Exploring overfitting in genetic programming. In: Liardet, P., Collet, P., Fonlupt, C., Lutton, E., Schoenauer, M. (eds.) EA 2003. LNCS, vol. 2936, pp. 267–277. Springer, Heidelberg (2004)
Poli, R., Langdon, W.B., McPhee, N.F.: A field guide to genetic programming. Lulu Enterprises, UK (2008)
Prechelt, L.: Early stopping - but when? In: Orr, G.B., Müller, K.-R. (eds.) NIPS-WS 1996. LNCS, vol. 1524, p. 55. Springer, Heidelberg (1998)
Thomas, J.D., Sycara, K.: The importance of simplicity and validation in genetic programming for data mining in financial data. In: Proceedings of the Joint GECCO 1999 and AAAI 1999 Workshop on Data Mining with Evolutionary Algorithms: Research Directions, pp. 7–11 (1999)
Tuite, C., Agapitos, A., O’Neill, M., Brabazon, A.: A Preliminary Investigation of Overfitting in Evolutionary Driven Model Induction: Implications for Financial Modelling. In: Di Chio, C., Brabazon, A., Di Caro, G.A., Drechsler, R., Farooq, M., Grahl, J., Greenfield, G., Prins, C., Romero, J., Squillero, G., Tarantino, E., Tettamanzi, A.G.B., Urquhart, N., Uyar, A.Ş. (eds.) EvoApplications 2011, Part II. LNCS, vol. 6625, pp. 120–130. Springer, Heidelberg (2011)
Tuite, C., Agapitos, A., O’Neill, M., Brabazon, A.: Early Stopping Criteria to Counteract Overfitting in Genetic Programming. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation. ACM, New York (2011) (forthcoming)
Vanneschi, L., Gustafson, S.: Using crossover based similarity measure to improve genetic programming generalization ability. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 1139–1146. ACM, New York (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Tuite, C., Agapitos, A., O’Neill, M., Brabazon, A. (2011). Tackling Overfitting in Evolutionary-Driven Financial Model Induction. In: Brabazon, A., O’Neill, M., Maringer, D. (eds) Natural Computing in Computational Finance. Studies in Computational Intelligence, vol 380. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23336-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-23336-4_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23335-7
Online ISBN: 978-3-642-23336-4
eBook Packages: EngineeringEngineering (R0)