Identifying Representation Bias in Large Language Models Used in Financial Sentiment Analysis

Financial sentiment analysis is the task of evaluating and quantifying the emotions and opinions expressed in financial news, reports, or social media to help investors and institutions make informed decisions. Financial institutions have been actively exploring the use of large language models (LLMs) to analyse market sentiment signals for a more nuanced understanding of a broader context. However, issues such as the scale of training data, model complexity, and the potential for human oversight can introduce or even amplify bias in these systems. Representation bias is a common challenge for LLMs as training data fail to properly represent the target groups, hence causes harmful bias in general-purpose use. Therefore, replacing current solutions with LLMs in financial organisations requires a robust evaluation methodology to ensure fairness. This paper investigates a three-level bias evaluation approach that specifically focuses on representation bias and presents a baseline evaluation of the FinBERT model. Step 1 uses a synthetic dataset that explicitly reveals sources of bias, structured as probability- and embedding-based evaluation recipes. Step 2 evaluates the model against data released by another country (e.g. Indian News dataset) to assess its performance in relation to more implicit biases. Step 3 examines individual problematic samples using token-based interpretability methods (e.g. integrated gradients). This paper presents the application of this structured bias evaluation process and its results on the FinBERT model. The evaluation code and dataset are available on GitHub (https://github.com/asabuncuoglu13/faid-test-financial-sentiment-analysis).