GCN-FinBERT-LSTM: a graph convolutional deep fusion model for multimodal stock market prediction

Abstract Time series forecasting models play a crucial role as decision support tools in various real-world domains. Among them, stock market represents a notably complex domain, characterized by a rapidly evolving temporal nature and a multitude of factors influencing stock prices. While many machine learning-based approaches for stock trend prediction have been proposed in the literature, they often focus on analyzing a single data source or modality. Additionally, those considering multiple modalities often do not consider correlations between different stocks, which limits their predictive power. In this paper, we introduce a novel multimodal deep fusion model for predicting stock trends. Our approach incorporates diverse data sources, including stock prices, technical indicators, graph-based stock interactions, and sentiment extracted from daily news headlines published by media outlets. Our model architecture consists of a Bidirectional Encoder Representations from Transformers (BERT) model branch fine-tuned on financial news, a long short-term memory (LSTM) branch, and a graph convolutional network (GCN) branch that captures salient temporal patterns in multi-stock data, encompassing stock prices and technical indicators. Our experiments, conducted on data from 17 comprehensive real-world stocks, reveal that our method is capable of effectively predicting market dynamics, outperforming popular baseline approaches, and yielding capital preservation capabilities in downtrend market conditions. Moreover, we showcase the effective trading performance of our model in portfolio analysis simulations, highlighting the positive impact of multimodal deep learning for stock trend prediction.