Is ChatGPT an Accurate Source of Information for Uninformed Taxpayers?

ABSTRACT In this study, we examine whether ChatGPT is an accurate source of information for uninformed taxpayers. During the 2023 and 2024 tax seasons, we entered common tax questions into ChatGPT models 3.5 and 4 and assessed the accuracy of the generated responses. In our main analyses, we find that ChatGPT’s overall percentage of correct responses ranged from 39 to 47 percent. ChatGPT also provides less accurate responses to tax questions that are more common, have more complex answers, require an evaluation of taxpayers’ fact patterns, and relate to tax information determined after ChatGPT’s knowledge cutoff date. However, we do not find that the ChatGPT model used (i.e., model 3.5 versus 4) impacts the accuracy of generated responses. Our study provides insights into the current capabilities of ChatGPT to provide accurate tax information and suggests that it is generally not a reliable source of tax guidance for uninformed taxpayers.