Comparing human and synthetic data in service research: using augmented language models to study service failures and recoveries

Purpose This study aims to explore Augmented Language Models (ALMs) for synthetic data generation in services marketing and research. It evaluates ALMs' potential in mirroring human responses and behaviors in service scenarios through comparative analysis with five empirical studies. Design/methodology/approach The study uses ALM-based agents to conduct a comparative analysis, leveraging SurveyLM (Bickley et al. , 2023) to generate synthetic responses to the scenario-based experiment in Söderlund and Oikarinen (2018) and four more recent studies from the Journal of Services Marketing . The main focus was to assess the alignment of ALM responses with original study manipulations and hypotheses. Findings Overall, our comparative analysis reveals both strengths and limitations of using synthetic agents to mimic human-based participants in services research. Specifically, the model struggled with scenarios requiring high levels of visual context, such as those involving images or physical settings, as in the Dootson et al. (2023) and Srivastava et al. (2022) studies. Conversely, studies like Tariq et al. (2023) showed better alignment, highlighting the model's effectiveness in more textually driven scenarios. Originality/value To the best of the authors’ knowledge, this research is among the first to systematically use ALMs in services marketing, providing new methods and insights for using synthetic data in service research. It underscores the challenges and potential of interpreting ALM versus human responses, marking a significant step in exploring AI capabilities in empirical research.