Artificial Intelligence-Based Prediction Modeling of Audit Dispositions in Korean Public Institutions

The Audit Report, prepared by public organizations through audits, contains information on the circumstances of the audit, the contents of the investigation, the problems identified during the investigation, and the results of the disposition. Despite the diversity of auditors and issues, these Audit Reports maintain a standardized and unified structure. In this study, the audit disposition results are categorized into 16 types, including notification, caution, and correction. These standardized Audit Reports support the expectation that similar content and wording can lead to similar results. With this in mind, this study developed a model that uses AI to predict the type of disposition in a standardized Audit Report. Since a single Audit Report may involve multiple dispositions depending on the audit situation and the number of people involved, the authors approached the problem as a multi-label classification issue and compared a deep learning-based 1D-CNN algorithm with a transformer-based BERT model. The 1D-CNN model demonstrated an accuracy of 75.3% when including the top result and 91.8% when including the top two results by audit type. Meanwhile, the BERT model showed an accuracy of 89.4% for the top results and 95.7% for the top two results. These models are currently operated within an enterprise audit system to support auditors' objective and rational decision-making.