Integrating structured and unstructured data for livestock price forecasting: a sustainability study from South Korea

Accurate forecasting of food prices is important for market regulation and long-term sustainability of the livestock industry. However, traditional forecasting methods often fail to consider unexpected external factors, such as disease outbreaks and natural disasters. Social media and online news have emerged as valuable sources for capturing these influences, but existing studies have primarily relied on short textual data, such as headlines or social media posts, which may lack depth and contextual richness. To address these challenges, we propose the Sentiment Analysis and Seasonal Decomposition (SASD) framework, a novel approach to enhance livestock price forecasting by integrating sentiment analysis of news data with seasonal decomposition of historical price trends. SASD framework, which systematically decomposes complex livestock price time series into trend, seasonal, and residual components, improving the forecasting accuracy by isolating seasonal patterns and irregular fluctuations. Additionally, we develop a Korean-language sentiment lexicon using an improved Term Frequency–Inverse Document Frequency (ITF-IDF) algorithm, enabling morpheme-level sentiment analysis for better sentiment extraction in Korean contexts. Furthermore, an attention-based Long Short-Term Memory (AM-LSTM) model enhances forecasting accuracy by prioritizing important sentiment shifts caused by unexpected external factors. To evaluate the effectiveness of our proposed framework, pork price and news data was collected between 2018 and 2021. The dataset comprises 14,588 news articles and corresponding price data across 874 days. Data from 2018 to 2020 were used for training, while data from 2021 served as the test set. For sentiment analysis, the ITF-IDF approach achieved F1-scores of 0.74 and 0.79 for negative and positive sentiment classifications, respectively. In terms of price prediction, the proposed AM-LSTM model outperformed traditional statistical methods, as well as machine learning and deep learning baselines, achieving improvements in MAE ranging from 43.0 to 87.4%. Furthermore, the SASD framework significantly reduced MAE in both short- and long-term predictions, by approximately 41.8, 60.7, 62.6, and 71.7% for 1-day, 7-day, 15-day, and 30-day forecasts, respectively. These results demonstrate that the SASD framework can be effectively implemented for livestock market analysis and has potential applicability beyond pork, including markets for lamb, beef, chicken, and other animal products.