Making IOAs work at scale: An evidence-based framework from six semesters of implementation

The widespread availability of generative artificial intelligence (GenAI) has increased demand for assessment approaches that provide stronger assurance of student learning while maintaining authentic educational value. Interactive Oral Assessments (IOAs) have emerged as a promising solution, but practical guidance for implementing them sustainably in large undergraduate subjects remains limited. This paper proposes a framework grounded in six consecutive semesters of continuous IOA implementation in a large undergraduate computing subject enrolling between 300 and 560 students per offering. Evidence draws on cohort mark distributions, IOA–examination correlations, academic integrity outcomes, and examiners’ consistency, as reflected in inter-rater reliability data. Repeated iterations of IOA design, delivery, evaluation, and refinement, informed by the literature and feedback from students and staff, produced four design principles critical to successful implementation of IOAs at scale. These principles address examiner calibration, rubric design, question depth and individual mark weighting. They are synthesised into an operational blueprint that covers assessment architecture, rubric design, question development, examiner preparation, and student preparation. Together, these provide transferable guidance for institutions seeking to implement IOAs sustainably at scale in the GenAI era.

Authors

Institutions

Publication Details

Journal
ASCILITE Publications
Published
2026-11-17
DOI
https://doi.org/10.65106/apubs.2026.3255
Primary Topic
Student Assessment and Feedback
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Making IOAs work at scale: An evidence-based framework from six semesters of implementation

Geela Venise Fabic Chee, Hasti Samadi
4 links
ASCILITE Publications
Student Assessment and Feedback
article

Making IOAs work at scale: An evidence-based framework from six semesters of implementation

Geela Venise Fabic Chee, Hasti Samadi
article en

Abstract

The widespread availability of generative artificial intelligence (GenAI) has increased demand for assessment approaches that provide stronger assurance of student learning while maintaining authentic educational value. Interactive Oral Assessments (IOAs) have emerged as a promising solution, but practical guidance for implementing them sustainably in large undergraduate subjects remains limited. This paper proposes a framework grounded in six consecutive semesters of continuous IOA implementation in a large undergraduate computing subject enrolling between 300 and 560 students per offering. Evidence draws on cohort mark distributions, IOA–examination correlations, academic integrity outcomes, and examiners’ consistency, as reflected in inter-rater reliability data. Repeated iterations of IOA design, delivery, evaluation, and refinement, informed by the literature and feedback from students and staff, produced four design principles critical to successful implementation of IOAs at scale. These principles address examiner calibration, rubric design, question depth and individual mark weighting. They are synthesised into an operational blueprint that covers assessment architecture, rubric design, question development, examiner preparation, and student preparation. Together, these provide transferable guidance for institutions seeking to implement IOAs sustainably at scale in the GenAI era.

ASCILITE Publications
The University of Melbourne (AU)
Quality education
Student Assessment and Feedback
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon