Abstract
This paper presents an application of the logistic smooth transition function and recurrent reinforcement learning for designing financial trading systems. We propose a trading system which is an upgraded version of the regime-switching recurrent reinforcement learning (RS-RRL) trading system referred to in the literature. In our proposed system (RS-RRL 2.0), we use an automated transition function to model the regime switches in equity returns. Unlike the original RS-RRL trading system, the dynamic of the transition function in our trading system is driven by utility maximization, which is in line with the trading purpose. Volume, relative strength index, price-to-earnings ratio, moving average prices from technical analysis, and the conditional volatility from a GARCH model are considered as possible options for the transition variable in RS-RRL type trading systems. The significance of Sharpe ratios, the choice of transition variables, and the stability of the trading system are examined by using the daily data of 20 Swiss SPI stocks for the period April 2009 to September 2013. The results from our experiment show that our proposed trading system outperforms the original RS-RRL and RRL trading systems suggested in the literature in terms of better Sharpe ratios recorded in three consecutive out-of-sample periods.
Chapter PDF
Similar content being viewed by others
Keywords
References
Allen, F., Karjalainen, R.: Using genetic algorithms to find technical trading rules. Journal of Financial Economics 51, 245–271 (1999)
Bertoluzzo, F., Corazza, M.: Making Financial Trading by Recurrent Reinforcement Learning. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007, Part II. LNCS (LNAI), vol. 4693, pp. 619–626. Springer, Heidelberg (2007)
Chan, K.S., Tong, H.: On estimating thresholds in autoregressive models. Journal of time series analysis 7, 179–190 (1986)
Creamer, G.: Model calibration and automated trading agent for euro futures. Quantitative Finance 12, 531–545 (2012)
Dempster, M.A.H., Leemans, V.: An automated FX trading system using adaptive reinforcement learning. Expert Systems with Applications 30, 543–552 (2006)
Gold, C.: FX trading via recurrent reinforcement learning. In: Proceedings of IEEE International Conference on Computational Intelligence in Financial Engineering, pp. 363–370. IEEE Computer Society Press, Los Alamitos (2003)
Gorse, D.: Application of stochastic recurrent reinforcement learning to index trading. In: European Symposium on Artificial Neural Networks (2011)
Graham, B., Zweig, J.: The Intelligent Investor: The Definitive Book on Value Investing. HarperCollins (2003)
Maringer, D., Ramtohul, T.: Regime-switching recurrent reinforcement learning for investment decision making. Computational Management Science 9, 89–107 (2012)
Maringer, D., Ramtohul, T.: Regime-switching recurrent reinforcement learning in automated trading. In: Brabazon, A., O’Neill, M., Maringer, D.G. (eds.) Natural Computing in Computational Finance, vol. 4, pp. 93–121. Springer (2012)
Maringer, D., Zhang, J.: Transition variable selection for regime switching recurrent reinforcement learning. In: Proceedings of the 2014 IEEE Conference on Computational Intelligence for Financial Engineering and Economics, pp. 407–412 (2014)
Moody, J., Wu, L., Liao, Y., Saffell, M.: Performance functions and reinforcement learning for trading systems and portfolios. Journal of Forecasting 17, 441–470 (1998)
Murphy, J.: Technical analysis of the financial markets: a comprehensive guide to trading methods and applications. New York Institute of Finance (1999)
Silva, A., Neves, R., Horta, N.: Portfolio optimization using fundamental indicators based on multi-objective ea. In: Proceedings of the 2014 IEEE Conference on Computational Intelligence for Financial Engineering and Economics, pp. 158–165 (2014)
Tong, H.: On a threshold model. In: Chen, C. (ed.) Pattern Recognition and Signal Processing. NATO ASI Series E: Applied Sc. (29), Sijthoff & Noordhoff, Netherlands (1978)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 IFIP International Federation for Information Processing
About this paper
Cite this paper
Zhang, J. (2014). Automating Transition Functions: A Way To Improve Trading Profits with Recurrent Reinforcement Learning. In: Iliadis, L., Maglogiannis, I., Papadopoulos, H. (eds) Artificial Intelligence Applications and Innovations. AIAI 2014. IFIP Advances in Information and Communication Technology, vol 436. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44654-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-662-44654-6_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44653-9
Online ISBN: 978-3-662-44654-6
eBook Packages: Computer ScienceComputer Science (R0)