The Written Self: Decoding Personality and Sex Differences Through Explainable AI

This study demonstrates how natural language processing and explainable AI techniques can advance our understanding of personality psychology, using sex differences in language as a case study. By analyzing stream-of-consciousness essays from 2,240 undergraduates using BERT with integrated gradients attribution, we identify distinct linguistic markers that characterize sex-specific trait expression. Our approach illustrates how explainability methods can help to detect systematic variations in language patterns across different trait levels and between males and females. Logistic regression analyses reveal Neuroticism as the strongest predictor of language typical of each sex (OR = 2.11, p ≤ .001), followed by Extraversion and Agreeableness. The model achieves 82.8\% accuracy in sex classification, and the explainability technique shows that sex differences in language often diminish at extreme trait levels. The study's methodological contribution lies in demonstrating how explainable AI can advance personality assessment by providing interpretable insights into the complex interaction between sex and personality in natural language use. The study also illustrates how explainable AI can advance personality assessment beyond descriptive statistics to provide interpretable insights into complex psychological phenomena, demonstrating a framework for deeper exploration of personality psychology findings.