Operational Use and AI Governance of LLM-Assisted Split Procurement Detection in University Internal Auditing

This study examines how large language model (LLM) assessments were used in an actual university internal audit. We analyzed system outputs, audit records, and accounts from two meetings with audit staff. A cloud implementation of a previously developed method processed 1,860,622 voucher line items covering 12 months. Candidate analysis covered eight organizational units. Across three monetary thresholds, 949 threshold-specific candidates were assigned to the three higher review-priority categories. After audit-specific filtering, auditors reviewed approximately 700 candidates, exported 36 for detailed examination, and selected four for departmental inquiry. They compared their own assessments with LLM-generated rationales and retained some uncertain candidates for further investigation. They also used priorities to decide how thoroughly to review candidates, depending on the volume in each department. Thus, LLM output informed both the interpretation of transactions and the allocation of review effort. These findings extend research on human oversight by showing how scrutiny of LLM rationales can coexist with reliance on priorities when allocating review effort. Independent evaluation of detection performance and time savings remains necessary.

Authors

Institutions

Publication Details

Journal
Systems
Published
2026-11-12
DOI
https://doi.org/10.3390/systems14111449
Primary Topic
Ethics and Social Impacts of AI
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Operational Use and AI Governance of LLM-Assisted Split Procurement Detection in University Internal Auditing

Takuo Suganuma, Shota Suzuki
Systems
Ethics and Social Impacts of AI
article

Operational Use and AI Governance of LLM-Assisted Split Procurement Detection in University Internal Auditing

Takuo Suganuma, Shota Suzuki
article en

Abstract

This study examines how large language model (LLM) assessments were used in an actual university internal audit. We analyzed system outputs, audit records, and accounts from two meetings with audit staff. A cloud implementation of a previously developed method processed 1,860,622 voucher line items covering 12 months. Candidate analysis covered eight organizational units. Across three monetary thresholds, 949 threshold-specific candidates were assigned to the three higher review-priority categories. After audit-specific filtering, auditors reviewed approximately 700 candidates, exported 36 for detailed examination, and selected four for departmental inquiry. They compared their own assessments with LLM-generated rationales and retained some uncertain candidates for further investigation. They also used priorities to decide how thoroughly to review candidates, depending on the volume in each department. Thus, LLM output informed both the interpretation of transactions and the allocation of review effort. These findings extend research on human oversight by showing how scrutiny of LLM rationales can coexist with reliance on priorities when allocating review effort. Independent evaluation of detection performance and time savings remains necessary.

SystemsVol. 14(11)
Tohoku University (JP), Tohoku University Cyberscience Center (JP)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon