Building Trust in Transformers with a Shap-Based Approach for Explaining Ai Decisions

Concerns regarding trust and interpretability are raised by the absence of explicit justifications for predictions made by transformer-based models. To address this problem, a Shapley Additive Explanations (SHAP)-based framework is implemented in our study to improve explainability by quantifying feature significance using Shapley values. The potential of SHAP in interpreting AI decisions, detecting biases, and improving transparency in model outputs is aimed to be evaluated, thereby promoting Explainable AI (XAI). A GPT-2 model is initialized, text inputs are processed in tokenized form, and SHAP is employed to derive feature values. Generated SHAP values are visualized via text plots highlighting the contribution of each input word to model predictions. Additionally, an experiment was conducted to examine biases in word associations, where implicit model preferences were revealed by input phrases. It was found that feature contributions are effectively identified by SHAP, with intuitive information into decision-making processes being provided by visualizations. When country names were included in input phrases, their effect on output word probabilities was examined. The predicted SHAP value for the output word “pasta” was determined to be −5.4053 when the input word was “Italians”, showing the strongest negative association. Compared to the universal base value of 10.3453, this represents a deviation of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$-47.74 \%$</tex>. This suggests the model embeds learned biases from training data, influencing its word associations. Quantifying the deviation of predicted values from the base value provides insights into how much one feature contributes to a model's decision, thereby exposing latent patterns and improving interpretability.