Mapping assessor judgement-in-action: A transition network analysis of dialogic moves in interactive oral assessment

Interactive Orals (IOs) are increasingly being adopted in higher education as an authentic and academically rigorous form of assessment. Despite the assessor’s central role in shaping the dialogic interaction, research has focused predominantly on student experience and assessment design, with limited empirical investigation of how assessors enact real-time dialogic practices. This study addresses this gap by examining assessor dialogue during real-time IOs, with attention to the functions and sequencing of assessors’ contributions. Drawing on audio-recorded and transcribed IO sessions from three courses across three institutions, involving seven assessors and 235 IO sessions in the larger study, the paper reports preliminary findings from 25 IOs involving two assessors with different levels of IO assessment experience. Assessor dialogue was coded into 13 substantive dialogic moves. Transition Network Analysis (TNA) was used to model transitions between dialogic moves and compared between experienced and novice assessors. Across 25 sessions, 723 assessor moves were coded. ACKNOWLEDGE was the most common dialogic move (30.4%), functioning as a central interactional hub. Six transitions significantly differed between the experienced and novice assessors. The largest difference was ELICIT to ACKNOWLEDGE, which was more probable for the experienced assessor, while ACKNOWLEDGE to ELICIT and REFRAME to REFRAME moves were more probable for the novice assessor. The initial findings suggest that assessor expertise may be expressed through how they take up student responses and orchestrate subsequent moves rather than the type of the questions. The paper proposes assessor dialogic orchestration as a useful conceptualisation of judgement-in-action and discusses implications for assessor training, moderation and assessment quality. However, the study does not establish that particular dialogic moves produce better student responses or assessment outcomes; these relationships require investigation in subsequent research.

Authors

Institutions

Publication Details

Journal
ASCILITE Publications
Published
2026-11-17
DOI
https://doi.org/10.65106/apubs.2026.3129
Primary Topic
Student Assessment and Feedback
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Mapping assessor judgement-in-action: A transition network analysis of dialogic moves in interactive oral assessment

Popi Sotiriadou, Gosia Ludwichowska, Joshua McLeod, Yuchen Liu et al.
2 links
ASCILITE Publications
Student Assessment and Feedback
article

Mapping assessor judgement-in-action: A transition network analysis of dialogic moves in interactive oral assessment

Popi Sotiriadou, Gosia Ludwichowska, Joshua McLeod, Yuchen Liu, Katrina Li
article en

Abstract

Interactive Orals (IOs) are increasingly being adopted in higher education as an authentic and academically rigorous form of assessment. Despite the assessor’s central role in shaping the dialogic interaction, research has focused predominantly on student experience and assessment design, with limited empirical investigation of how assessors enact real-time dialogic practices. This study addresses this gap by examining assessor dialogue during real-time IOs, with attention to the functions and sequencing of assessors’ contributions. Drawing on audio-recorded and transcribed IO sessions from three courses across three institutions, involving seven assessors and 235 IO sessions in the larger study, the paper reports preliminary findings from 25 IOs involving two assessors with different levels of IO assessment experience. Assessor dialogue was coded into 13 substantive dialogic moves. Transition Network Analysis (TNA) was used to model transitions between dialogic moves and compared between experienced and novice assessors. Across 25 sessions, 723 assessor moves were coded. ACKNOWLEDGE was the most common dialogic move (30.4%), functioning as a central interactional hub. Six transitions significantly differed between the experienced and novice assessors. The largest difference was ELICIT to ACKNOWLEDGE, which was more probable for the experienced assessor, while ACKNOWLEDGE to ELICIT and REFRAME to REFRAME moves were more probable for the novice assessor. The initial findings suggest that assessor expertise may be expressed through how they take up student responses and orchestrate subsequent moves rather than the type of the questions. The paper proposes assessor dialogic orchestration as a useful conceptualisation of judgement-in-action and discusses implications for assessor training, moderation and assessment quality. However, the study does not establish that particular dialogic moves produce better student responses or assessment outcomes; these relationships require investigation in subsequent research.

ASCILITE Publications
Griffith University (AU), Deakin University (AU), La Trobe University (AU), Adelaide University (AU), Monash University (AU), The University of Adelaide (AU)
Quality education
Student Assessment and Feedback
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon