Trading with confidence: comprehensive uncertainty estimation for reinforcement learning agents

Abstract

Reinforcement Learning (RL) has emerged as a powerful approach in financial trad- ing, enabling agents to learn optimal strategies through direct market interaction. However, financial markets are highly uncertain, mainly due to price fluctuations driven by stochastic volatility, model limitations, and regime shifts. Traditional RL models struggle in dynamic environments, often failing to adapt to sudden market disruptions, and thus leading to suboptimal trading decisions. To address this challenge, we propose an uncertainty-aware RL framework that integrates distributional, epistemic, and aleatoric uncertainty estimations. Our approach enhances uncertainty estimation using SHAP-weighted reconstruction uncertainty, MC Dropout, and an LSTM-based technical indicator consensus mechanism. Experimental results on five major U.S. stock indices demonstrate that RL agents equipped with uncertainty estimation significantly outperform traditional models in return and risk management. This study advances uncertainty estimation in RL-based financial trading, with future research extending its application to other asset classes and alternative RL architectures for greater adaptability.