Reinforcement trading for multi-market portfolio with crisis avoidance

Abstract

The global financial market comes to a new crisis in 2020 triggered by the COVID-19 pandemic. During such a period, it is crucial for a portfolio manager to adopt policies that can preserve the value of the portfolio. Although innovations in computational finance using Machine Learning emerge rapidly, many of the works are using off-line supervised learning that is dependent on the training data in a specific period, and the model is not capable of direct trading. Additionally, many other works using Reinforcement Learning approaches are built with in-house tools and is lack of extensibility. As such, these models are neither transferrable to greater markets in a longer time range, nor are they capable to handle the black swan or grey rhino events that reappear almost every decade. In this paper, we proposed a Reinforcement Learning trading framework with a crisis avoidance algorithm. The framework adopts the open-sourced OpenAI Gym standard and Stable Baseline model that are open for third-party tools and future extension. We invented a Reinforcement Learning Environment to describe the market behavior with technical analysis and finite rule-based action sets. The framework further implements a crisis detection and avoidance algorithm. The experiment result shows that the models trained by the framework performed as well as buy-and-hold strategy benchmark in the bullish period of 2015-2019. Furthermore, very much accredited to the crisis avoidance algorithm, the models acted 17% better than buy-and-hold during all testing windows no less than 5 years in 2000-2019.