ChatGPT as digital accounting assistant: evaluating output performance in financial accounting tasks

Purpose The study aims to analyze ChatGPT’s performance in financial accounting judgment tasks, focusing on variations by task category, model version, prompting method and accounting standards. Design/methodology/approach The authors conducted an experimental design using 480 prompts across four task categories, six GPT models, two prompting techniques and two accounting standards [International Financial Reporting Standards (IFRS) and Austrian generally accepted accounting principles (GAAP)]. Experts rated outputs, which were analyzed using logistic regression and predictive machine learning. Findings The results indicate that the GPT model has a strong influence on whether the response is correct, with newer models performing better. The task category is a major factor, with GPT models struggling especially with rule-based calculations that are straightforward for human professionals. Performance is significantly lower for Austrian GAAP compared to IFRS, likely due to limited training data. Prompting has a limited overall impact. Research limitations/implications The findings reveal ChatGPT’s potential and limits in accounting judgment, implying research needs for task delegation and the integration of frameworks. Practical implications While ChatGPT demonstrates potential as a digital assistant for specific tasks (with an accuracy up to 77%), expert oversight remains critical to ensure output performance. The presence of false positives can create a misleading sense of security, underscoring the need for professional judgment in compliance-sensitive contexts. Originality/value This study introduces a novel task taxonomy based on the Dreyfus model to assess ChatGPT’s performance in accounting judgment. It uniquely combines task complexity, accounting standards and model architecture to identify key performance drivers and frames ChatGPT through the lens of functional agency, offering a new perspective on its professional applicability in financial accounting judgment.