Age-inclusive AI assessments: testing and refining open-ended prompts with human and LLM-simulated data

As workforces become increasingly age-diverse, organizations are increasingly adopting AI-based tools, such as automated video interviews (AVIs) and chatbots, to evaluate job applicants. Although prior research has documented age-related differences in communication styles, we know less about how these differences appear in AI-based assessments or whether the design of the assessment can reduce them. Across two studies, we examined age-related differences in how people responded to AVI prompts (i.e., interview questions) and whether changing the wording of prompts could make these assessments more age inclusive. In Study 1, we compared linguistic patterns in AVI responses among workers aged 18 to 66 + years or older. After controlling for education, gender, race, SES, and self-reported personality and vocational interest scores, the findings revealed significant age-related differences in semantic content (i.e., what people talked about) and language complexity (i.e., how complex their language was). In Study 2, we used ChatGPT to simulate AI personas representing workers of different ages and used these personas to test revised AVI prompts. Results demonstrated that prompt wording can meaningfully influence the magnitude and pattern of age-related linguistic differences, with some differences attenuated and others maintained. These findings suggest that prompt design may play an important role in making AI-based assessments more inclusive. We discuss the implications of the findings for understanding age-related differences in language and for designing fairer and more valid AI-based assessment tools.