Making Feedback Quality Visible to Instructors: An AI-Supported Learning Analytics Dashboard Linking Peer Code Review to Pedagogical Action in Programming Education

Peer code review is widely used in programming and software engineering education, but its instructional value depends on the quality of student feedback rather than review completion alone. In large classes, instructors can readily monitor participation but lack scalable methods for assessing whether feedback is relevant, concrete, and constructive, limiting opportunities for timely instructional intervention. To address this gap, this study developed and evaluated the Review Engagement Graph (REG), an instructor focused learning analytics dashboard that integrates multilabel artificial intelligence classification of feedback quality with interactive network visualization. REG operationalizes feedback quality across three independent dimensions and translates five recurring visual patterns into corresponding pedagogical actions while preserving instructor judgment. The system was evaluated in two studies. Study 1 compared seven classification models spanning traditional machine learning, deep learning, fine-tuned transformers, and prompted large language models using 2,398 peer review comments independently annotated by two experts. Fine-tuned RoBERTa achieved the strongest overall performance, with a macro F1 score of 0.828. Study 2 evaluated REG with instructors and teaching assistants through scenario-based sessions. REG achieved a mean System Usability Scale score of 81.13, exceeding the benchmark of 68, and a mean perceived usefulness score of 4.31 out of 5. These findings demonstrate that AI classified feedback quality indicators can be transformed into actionable visual evidence through a pattern to action framework that supports timely and informed instructor decision making.

Authors

Publication Details

Journal
International Journal of Software Engineering and Knowledge Engineering
Published
2026-10-28
DOI
https://doi.org/10.1142/s0218194026500968
Primary Topic
Online Learning and Analytics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Making Feedback Quality Visible to Instructors: An AI-Supported Learning Analytics Dashboard Linking Peer Code Review to Pedagogical Action in Programming Education

Hsi‐Min Chen, Hoang‐Thanh Duong, Duong Thuy Trang Nguyen, Min-Jie Chung
3 links
International Journal of Software Engineering and Knowledge Engineering
Online Learning and Analytics
article

Making Feedback Quality Visible to Instructors: An AI-Supported Learning Analytics Dashboard Linking Peer Code Review to Pedagogical Action in Programming Education

Hsi‐Min Chen, Hoang‐Thanh Duong, Duong Thuy Trang Nguyen, Min-Jie Chung
article en

Abstract

Peer code review is widely used in programming and software engineering education, but its instructional value depends on the quality of student feedback rather than review completion alone. In large classes, instructors can readily monitor participation but lack scalable methods for assessing whether feedback is relevant, concrete, and constructive, limiting opportunities for timely instructional intervention. To address this gap, this study developed and evaluated the Review Engagement Graph (REG), an instructor focused learning analytics dashboard that integrates multilabel artificial intelligence classification of feedback quality with interactive network visualization. REG operationalizes feedback quality across three independent dimensions and translates five recurring visual patterns into corresponding pedagogical actions while preserving instructor judgment. The system was evaluated in two studies. Study 1 compared seven classification models spanning traditional machine learning, deep learning, fine-tuned transformers, and prompted large language models using 2,398 peer review comments independently annotated by two experts. Fine-tuned RoBERTa achieved the strongest overall performance, with a macro F1 score of 0.828. Study 2 evaluated REG with instructors and teaching assistants through scenario-based sessions. REG achieved a mean System Usability Scale score of 81.13, exceeding the benchmark of 68, and a mean perceived usefulness score of 4.31 out of 5. These findings demonstrate that AI classified feedback quality indicators can be transformed into actionable visual evidence through a pattern to action framework that supports timely and informed instructor decision making.

International Journal of Software Engineering and Knowledge Engineering
Quality education
Openalex Percentile: Top 7%
Online Learning and Analytics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon