Multivariate forecasting on financial time series with transformer model

Abstract

The stock market provides an open platform for companies to raise capital for funding necessary operations and expanding their business, which is essential in driving world economic growth. Given its importance for making profitable trading decisions, stock price prediction has always been an active area of interest among investors and quantitative researchers. Although various machine learning models have been used extensively for stock prediction, these techniques suffer from several drawbacks which make them unsuitable for modelling financial time series. Another limitation of existing studies is that they focus primarily on single-stock prediction and did not consider the information of other stocks during forecasting. As the prices of multiple assets are often correlated, this approach is insufficient because it disregards the underlying relations between stocks.

To address such challenges, this thesis proposes the use of a Transformer for stock price forecasting. Given its impressive performance in Natural Language Processing tasks, the Transformer is adapted for processing similarly sequential elements in time series. At the core of the Transformer are attention mechanisms, which learn the dependencies in a sequence and rank the importance of related elements. Attention eliminates the need for sequential computation, which allows for much shorter training time and more effective handling of extremely long sequences. Furthermore, this paper builds upon prior works by implementing single-output and multi-output prediction scenarios with or without external stock predictors. For every stock, technical indicators were extracted and used as model input together with basic stock information.

The proposed Transformer was evaluated on S&P 500 stocks from various sectors and compared with several baseline models, namely the Long Short-Term Memory (LSTM), Support Vector Regression (SVR), Random Forest and Categorical Boosting (CatBoost). Based on the experimental results, using external stock predictors and multi-output prediction do not necessarily guarantee an improvement in performance. However, it is clear that the Transformer achieves superior performance over the baselines, and demonstrates great potential for time series forecasting.