Crude oil price fluctuation forecasting incorporating news sentiment based on improved sentiment lexicon

To effectively and accurately capture future fluctuations in crude oil prices from massive news data, this study proposes a forecasting method based on an improved sentiment lexicon and integrating news sentiment. Firstly, a rule-based approach was proposed to obtain a news collection containing price fluctuation information. Secondly, an automatic algorithm for constructing an improved sentiment lexicon for oil price forecasting was proposed. The algorithm first updates the sentiment polarity of some words in the Loughran&McDonald (LM) financial lexicon, combined with the LM oil price lexicon, to establish an expanded seed lexicon (LM-S), and then extends LM-S to form the final domain specific sentiment lexicon. Thirdly, a news classification model was constructed, dividing news into eight categories and extracting emotional features comprehensively through four types of emotional indicators. Finally, by integrating WTI crude oil prices, sentiment indicators, and financial indicators, and using feature screening and lag calculation, LSTM, BP, and GRU models are used to predict future price fluctuations of crude oil. In the experiment, directional accuracy, MAPE, and the newly proposed directional absolute error rate were used as evaluation indicators. Shapley values were used to explain the contribution of each feature to the forecasting results.