Mind the Gap: Gender-based Differences in Occupational Embeddings
Large Language Models (LLMs) offer promising alternatives to traditional occupational coding approaches in survey research.Using a German dataset, we examine the extent to which LLM-based occupational coding differs by gender.Our findings reveal systematic disparities: gendered job titles (e.g., "Autor" vs. "Autorin", meaning "male author" vs. "female author") frequently result in diverging occupation codes, even when semantically identical.Across all models, 54%-82% of gendered inputs obtain different Top-5 suggestions.The practical impact, however, depends on the model.GPT includes the correct code most often (62%) but demonstrates female bias (up to +18 pp).IBM is less accurate (51%) but largely balanced.Alibaba, Gemini, and MiniLM achieve about 50% correct-code inclusion, and their small (< 10 pp) and direction-flipping gaps could indicate a sampling noise rather than gender bias.We discuss these findings in the context of fairness and reproducibility in NLP applications for social data.
Authors
- Olga Kononykhina
- Anna‐Carolina Haensch (ORCID: https://orcid.org/0000-0001-6772-0393)
- Frauke Kreuter (ORCID: https://orcid.org/0000-0002-7339-2645)
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.gebnlp-1.7
- Primary Topic
- Gender Diversity and Inequality
- Type
- article
- Field-Weighted Citation Impact
- 0.00