Data-Driven Insights: Leveraging Sentiment Analysis and Latent Profile Analysis for Financial Market Forecasting

Background: This study explores an innovative integration of big data analytics techniques aimed at enhancing predictive modeling in financial markets. It investigates how combining sentiment analysis with latent profile analysis (LPA) can accurately forecast stock prices. This research aligns with big data methodologies by leveraging automated content analysis and segmentation algorithms to address real-world challenges in data-driven decision-making. This study leverages advanced computational methods to process and segment large-scale unstructured data, demonstrating scalability in data-rich environments. Methods: We compiled a corpus of 3843 financial news articles on Teva Pharmaceuticals from Bloomberg and Reuters. Sentiment scores were generated using the VADER tool, and LPA was applied to identify eight distinct sentiment profiles. These profiles were then used in segmented regression models and Structural Equation Modeling (SEM) to assess their predictive value for stock price fluctuations. Results: Six of the eight latent profiles demonstrated significantly higher predictive accuracy compared to traditional sentiment-based models. The combined profile-based regression model explained 47% of the stock price variance (R2 = 0.47), compared to 10% (R2 = 0.10) in the baseline model using sentiment analysis alone. Conclusion: This study pioneers the use of latent profile analysis (LPA) in sentiment analysis for stock price prediction, offering a novel integration of clustering and financial forecasting. By uncovering complex, non-linear links between market sentiment and stock movements, it addresses a key gap in the literature and establishes a powerful foundation for advancing sentiment-based financial models.