MiniBert24: A Lightweight Transformer-Based Model for Stock Market Movement Prediction

The rapid growth of financial markets has necessitated developing more precise and efficient frameworks to predict stock price movements. This paper introduces MiniBert24, a lightweight, transformer-based architecture specifically designed for stock market movement prediction by leveraging news related to finance and past stock price information. The model is tailored for high computational efficiency, combining a reduced number of transformer layers with fewer attention heads while retaining strong predictive power. We utilize a unique dataset incorporating historical headlines from the Reddit WorldNews channel and Dow Jones Industrial Average (DJIA) data, creating a binary classification task to forecast daily stock price movement. MiniBert24 obtains an excellent AUC of 96.7% and an F1-score of 96.43%, outperforming standard transformer models while reducing computational overhead. These results demonstrate the potential of MiniBert24 to serve as a robust tool for market prediction in real-time trading environments. The model’s ability to fuse textual data with stock market indicators provides a compelling solution for financial analysts and algorithmic traders seeking efficient, high-performing predictive models.