Synthetic Voices: Evaluating the Fidelity of LLM-Generated Personas in Representing People’s Financial Wellbeing

Large Language Models (LLMs) can impersonate the writing style of authors, characters, and groups of people, but can these personas represent their opinions?If so, it creates opportunities for businesses to obtain early feedback on ideas from a synthetic customerbase.In this paper, we test whether LLM synthetic personas can answer financial wellbeing questions similarly to the responses of a financial wellbeing survey of more than 3,500 Australians.We focus on identifying salient biases of 765 synthetic personas using four state-of-the-art LLMs built over 35 categories of personal attributes.We noticed clear biases related to age, and as more details were included in the personas, their responses increasingly diverged from the survey toward lower financial wellbeing.With these findings, it is possible to understand the areas in which creating synthetic LLM-based customer personas can yield useful feedback for faster product iteration in the financial services industry and potentially other industries.