Towards AI-assisted validation of university course profiles: A work-in-progress framework for policy compliance
Reviewing university course profiles against institutional policies remains a largely manual process that is time-consuming and repetitive. This paper presents the design of an AI-assisted framework for course profile university policy compliance checking that combines Large Language Model (LLM)-based semantic analysis with deterministic rule-based validation, together with the initial implementation and evaluation of its LLM-based validation component. A prototype using Claude Sonnet 4.5 was developed and evaluated though an exploratory case study on a published course profile, using a baseline scenario and three modified versions with deliberately introduced policy violations. Although the LLM was able to recognise a stated policy exception occasionally, it produced false positives on the compliant baseline course profile, failed to detect a numerical error, and returned inconsistent validation outcomes for the same input. These findings show that LLM-only validation is not sufficiently reliable, motivating a combined approach that integrates deterministic rule-based validation with an LLM-based agentic workflow. This work represents the first stage of a broader study investigating hybrid AI-assisted approaches to course profile policy validation in higher education.
Authors
- Lizzie Li
- Aneesha Bakharia (ORCID: https://orcid.org/0000-0002-5802-1280)
- Xintong Tan
- Hui Zhou
Institutions
- The University of Queensland (AU)
Publication Details
- Journal
- ASCILITE Publications
- Published
- 2026-11-17
- DOI
- https://doi.org/10.65106/apubs.2026.3317
- Primary Topic
- Artificial Intelligence in Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00