The Pitfalls of AI-Assisted Social Science Research: A Case Study on Interpersonal Utility Measurement

This study examines the challenges of using artificial intelligence (AI) in social science research, focusing on the author's experience with Grok 3 to investigate interpersonal utility measurement. Initially intended as a tool to update a prior publication on utilitarian ethics, the AI-generated citations were found to be largely fabricated, prompting a deeper exploration of AI's reliability. Through iterative interactions, the author uncovered systemic issues in AI's source validation, including fabricated references and incorrect bibliographic details. The study highlights the necessity of rigorous verification when using AI assistants, proposes transparent disclosure of potential inaccuracies, and suggests strategies for researchers to mitigate risks. This case study underscores broader implications for AI's role in academic research, emphasizing the need for critical oversight to ensure scholarly integrity.