Application of Deep Reinforcement Learning for Stock Trading on The Indonesia Stock Exchange

Deni Saepudin; Khalifatur  Rauf

doi:10.23887/janapati.v14i1.83775

Authors

Deni Saepudin School of Computing, Telkom University
Khalifatur Rauf School of Computing, Telkom University

DOI:

https://doi.org/10.23887/janapati.v14i1.83775

Keywords:

Deep Reinforcement Learning, Stock Trading, Automated Trading Systems, Advantage Actor-Critic, Proximal Policy Optimization

Abstract

In the last couple of years, stock trading has gained so much popularity because of its promising returns. However, most investors do not pay attention to the risks of trading without analysis, which can lead to a big loss. Some to reduce these risks, try their luck with automated and pre-programmed trading systems, which are called Expert Advisors. The current study examines the application of DRL for automated assistance in trading with an emphasis on decision-making enhancement, particularly the use of DRL in order to realize high asset returns with a low risk of exposure. Concretely, the two applied DRL methods within this work are A2C and PPO. By systematic testing, the A2C method produced a Sharpe Ratio of 1.6009 with a cumulative return of 1.4468, while the PPO method achieved a Sharpe Ratio of 1.7628 with a cumulative return of 1.4767. These were fine-tuned for the most optimal learning rates, cut loss, and take profit ratios, thus showing great promise with the capability to tune up trading strategies and improve trading performances. The research leverages these DRL techniques, hence arriving at better trading strategies that balance profit and risk, while underlining the promise of advanced algorithms in automated stock trading.

References

Z. Xiong, X.-Y. Liu, S. Zhong, H. Yang, and A. Walid, “Practical Deep Reinforcement Learning Approach for Stock Trading,” CoRR, vol. abs/1811.07522, 2018, [Online]. Available: http://arxiv.org/abs/1811.07522

G. Jeong and H. Y. Kim, “Improving financial trading decisions using deep Q-learning: Predicting the number of Shares, action Strategies, and transfer learning,” Expert Syst Appl, vol. 117, 2019, doi: 10.1016/j.eswa.2018.09.036.

Y. Li, W. Zheng, and Z. Zheng, “Deep Robust Reinforcement Learning for Practical Algorithmic Trading,” IEEE Access, vol. 7, 2019, doi: 10.1109/ACCESS.2019.2932789.

Z. Liang, H. Chen, J. Zhu, K. Jiang, and Y. Li, “Adversarial Deep Reinforcement Learning in Portfolio Management,” 2018. [Online]. Available: https://arxiv.org/abs/1808.09940

S. Yang, S. Gao, and X. Liu, “An improved deep reinforcement learning algorithm for stock trading strategies,” Appl Soft Comput, vol. 81, p. 105735, 2019.

X. Li, H. Xie, R. Wang, and Y. Cai, “Reinforcement learning for optimal trading strategies in the stock markets,” Expert Syst Appl, vol. 159, p. 113620, 2020.

Y. Chen, J. Gao, and L. Wang, “Automated stock trading with deep reinforcement learning,” IEEE Trans Neural Netw Learn Syst, vol. 32, no. 6, pp. 2231–2242, 2021.

B. Liu, Y. Yang, and S. Wang, “Risk management in stock trading with deep reinforcement learning,” Journal of Financial Markets, vol. 68, p. 100693, 2023.

V. Mnih et al., “Asynchronous methods for deep reinforcement learning,” in 33rd International Conference on Machine Learning, ICML 2016, 2016.

T. P. Lillicrap et al., “Continuous control with deep reinforcement learning,” in 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, 2016.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” CoRR, vol. abs/1707.06347, 2017, [Online]. Available: http://arxiv.org/abs/1707.06347

I. Grondman, L. Busoniu, G. A. D. Lopes, and R. Babuška, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,” 2012. doi: 10.1109/TSMCC.2012.2218595.

D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” in 31st International Conference on Machine Learning, ICML 2014, 2014.

C. Yin, S. Wang, and L. Zha, “Stock trading strategy based on CNN and LSTM,” in Proceedings of the 2019 8th International Conference on Computing and Pattern Recognition, 2019, pp. 1–6.

J. Patel, S. Shah, P. Thakkar, and K. Kotecha, “Predicting stock market index using fusion of machine learning techniques,” Expert Syst Appl, vol. 42, no. 4, pp. 2162–2172, 2020.

X. Liu and X. Wang, “An ensemble deep learning model for stock prediction based on stock prices and news sentiment,” IEEE Access, vol. 9, pp. 130254–130272, 2021.

Y. Zhang and Y. Zhang, “A novel stock trading strategy based on the improved reinforcement learning algorithm,” Journal of Financial Markets, vol. 58, p. 100673, 2022.

X. Chen and Y. Zhou, “Stock trading model using hybrid reinforcement learning techniques,” Appl Soft Comput, vol. 118, p. 108475, 2023.

N. Yashina, O. Kashina, S. Yashin, N. Pronchatova-Rubtsova, and I. Khorosheva, “Digital Methods of Technical Analysis for Diagnosis of Crisis Phenomena in the Financial Market,” International Journal of Technology, vol. 13, no. 7, pp. 1403–1411, 2022, doi: 10.14716/ijtech.v13i7.6187.

T. Kabbani and E. Duman, “Deep Reinforcement Learning Approach for Trading Automation in The Stock Market,” Jul. 2022, doi: 10.1109/ACCESS.2022.3203697.