Application of machine learning for stock index forecast

Abstract

Stock Prediction has always been a popular area of research. However, in the last decade, the advances in machine learning has brought about new possibilities with new algorithms and models that can be utilized in stock prediction. With the newfound interest, it sparked a growing amount of research into the subject.

This project focuses on a sole index, New York Stock Exchange Composite (NYA) as the subject of research. Both technical and content features are mined and fed to a machine learning model to predict the price movement of NYA. Content features were obtained from selected accounts of the popular social media site, Twitter.

The proposed model includes a 2-layer Long-Short Term Memory (LSTM) network as its basis. Content features are preprocessed, then sentiments are extracted with the use of several probabilistic algorithms and fed into the network with the technical features. The proposed model was applied and evaluated in comparison with a benchmark model and the models with various probabilistic algorithms for sentiment analysis.

The results of the project have concluded that the use of sentiment analysis of twitter news has improved the prediction accuracy and performance of the model sufficiently; however improvement varies with the types and combination of algorithms used.