A study of reinforcement learning in quantitative trading

Abstract

In this project, we explore using Reinforcement Learning (RL) for Gamma Scalping in options trading to design a dynamic hedging strategy with maximum risk-adjusted returns. Traditional Gamma Scalping methods employ pre-defined heuristics and static models that are unable to deal with dynamic market conditions. To address this limitation, we introduce a hedge agent that uses a multi-layer perceptron (MLP) and a long short-term memory (LSTM) to manage high-dimensional market information (1071 features) and generate sequential hedging decisions based on Delta values and context features. Our approach employs distribution shift detection techniques to monitor market condition changes. With Brier Score, Label Shift, and Covariate Shift as metrics, we assess the trained model's validity and determine if re-training is necessary to maintain stable performance. Empirical evaluations on Moontower-simulated data demonstrate that our RL-based approach enhances annualized return and Sharpe ratio and reduces drawdown and hedging cost. It highlights RL's ability to be applied to dynamic options trading risk management and provides insight into how models can be adaptively updated given changing market conditions.