Investor sentiment from images: a few-shot learning investigation

Purpose This research aims to extract emotional features from New York Times news images (2018–2023) using few-shot learning approaches. Leveraging machine learning, it offers a systematic investigation into how image-driven emotions affect investor behavior in the US equity market and contribute to the prediction of market movements. Design/methodology/approach This study employs the DeepEMD model to extract emotional features from 181,233 news images, constructing a daily sentiment index based on visual media. By defining sentiment thresholds, the study develops differentiated strategies for positive and negative emotional signals. In addition, it integrates four machine learning models – AdaBoost, Support Vector Machine, ExtraTrees and Random Forest – alongside a traditional linear regression model to forecast the prices of various US stock market indices. Findings This study finds that news image sentiment has a significant impact on financial markets. Positive sentiment strategies applied to serious news topics are associated with higher returns, whereas negative sentiment in entertainment-related content signals potential opportunities for contrarian investment. Moreover, the influence of image-based sentiment on the market exhibits a delayed effect of approximately 2–3 days, with particularly strong predictive power for small-cap stocks. Compared with the traditional linear models, machine learning approaches demonstrate superior performance in capturing the nonlinear dynamics between sentiment and market behavior, offering novel analytical tools for behavioral finance research and sentiment-driven anomaly-based investment strategies. Originality/value This study integrates visual data analysis into the domain of behavioral finance, highlighting the distinctive role of image-based sentiment in uncovering market anomalies and informing investment strategies.