A deep reinforcement learning approach to automated stock trading

Abstract

Deep reinforcement learning is recognised as an advantageous solution to automated stock trading. It has the potential of establishing an end-to-end solution that directly generate the target portfolio from market data. But applying it to financial tasks often undergoes an error-pone development process. This project starts with a careful survey on deep reinforcement learning and its recent applications in financial tasks. The actor-critic approach is the most common one and fits well with the stochastic and complex nature of stock market. Four state-of-the-art algorithms, DDPG, A2C, PPO, and TD3, are studied in detail. A deep reinforcement learning approach to automated stock trading is designed and implemented. In this project, the trading simulator is configured with preprocessed historical market data, the trading strategy is learnt from neural networks, and the trading performance is evaluated via automated backtesting. Besides, the transaction cost and risk-aversion are incorporated. The project is featured with completeness and scalability. It covers all components required in the development process. Various financial factors are extracted from market data. The trading simulator is compatible with trading agents in arbitrary structures and provides three rewards function for adjustments to different markets. And the trading agents are trained by fine-tuned algorithms and shown to outperform the baseline trading strategies. The achievement of this project is that it streamlines the development process of applying deep reinforcement learning to automated stock trading and provide guidance for future works.