Impact of AI-assisted programming on a data-driven learning analytics framework for detecting the performance capability gap and cognitive debt in programming education

Abstract This study examines a performance discrepancy in programming education: students often score higher on AI-assisted coding tasks than on unaided conceptual assessments of similar material. Drawing on the construct of cognitive debt, operationally defined here as the discrepancy between a student’s AI-assisted output and the same student’s ability to demonstrate the underlying concepts without AI assistance, we quantify this gap among 100 students (of 111 enrolled; 50 male, 50 female) in a Bangladeshi university programming course, using an explanatory sequential mixed-methods design. Two performance measures—an AI-assisted practical coding test and an unaided conceptual test—were compared using paired-samples testing, correlation and regression, exploratory k-means cluster analysis (k = 4), and CGPA and gender covariate analysis, supplemented by semi-structured interviews with 10 purposively selected students. Students scored substantially higher on the AI-assisted test than the unaided test. The two measures were only moderately correlated, and cluster analysis identified four exploratory performance profiles, including a Cognitive Debt Zone comprising 34% of students who combined high AI-assisted scores with low conceptual scores. CGPA was positively associated with performance on both tests (r = .55 for each) but negatively associated with the size of the AI-assisted/unaided gap; no significant gender differences emerged, though the study was underpowered to rule out small effects. These findings indicate a measurable performance gap between AI-assisted and unaided assessment that is consistent with a hidden shortfall in conceptual understanding. Because the design is cross-sectional, the two tests differ in format and difficulty, and there is neither a no-AI practical control nor a pre-AI baseline, the study cannot show that skills have accumulated or eroded, cannot attribute the gap to AI use alone, and does not provide a directly validated measure of cognitive debt itself. Building on these results, we propose the AI-Assisted Task–Feynman Technique Cognitive Debt Framework, a preliminary diagnostic and monitoring tool based on these performance-gap indicators, and discuss implications for the quality of AI-integrated programming education (SDG 4).

Authors

Institutions

Publication Details

Journal
Discover Sustainability
Published
2026-11-15
DOI
https://doi.org/10.1007/s43621-026-05285-0
Primary Topic
Teaching and Learning Programming
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Impact of AI-assisted programming on a data-driven learning analytics framework for detecting the performance capability gap and cognitive debt in programming education

Md. Rakibul Hoque, Mohammad Rakibul Islam Bhuiyan, M. Helal Uddin Ahmed, Md Shawan Mia et al.
Discover Sustainability
Teaching and Learning Programming
article

Impact of AI-assisted programming on a data-driven learning analytics framework for detecting the performance capability gap and cognitive debt in programming education

Md. Rakibul Hoque, Mohammad Rakibul Islam Bhuiyan, M. Helal Uddin Ahmed, Md Shawan Mia, Nuzhat Nuyeri Nuheen
article en

Abstract

Abstract This study examines a performance discrepancy in programming education: students often score higher on AI-assisted coding tasks than on unaided conceptual assessments of similar material. Drawing on the construct of cognitive debt, operationally defined here as the discrepancy between a student’s AI-assisted output and the same student’s ability to demonstrate the underlying concepts without AI assistance, we quantify this gap among 100 students (of 111 enrolled; 50 male, 50 female) in a Bangladeshi university programming course, using an explanatory sequential mixed-methods design. Two performance measures—an AI-assisted practical coding test and an unaided conceptual test—were compared using paired-samples testing, correlation and regression, exploratory k-means cluster analysis (k = 4), and CGPA and gender covariate analysis, supplemented by semi-structured interviews with 10 purposively selected students. Students scored substantially higher on the AI-assisted test than the unaided test. The two measures were only moderately correlated, and cluster analysis identified four exploratory performance profiles, including a Cognitive Debt Zone comprising 34% of students who combined high AI-assisted scores with low conceptual scores. CGPA was positively associated with performance on both tests (r = .55 for each) but negatively associated with the size of the AI-assisted/unaided gap; no significant gender differences emerged, though the study was underpowered to rule out small effects. These findings indicate a measurable performance gap between AI-assisted and unaided assessment that is consistent with a hidden shortfall in conceptual understanding. Because the design is cross-sectional, the two tests differ in format and difficulty, and there is neither a no-AI practical control nor a pre-AI baseline, the study cannot show that skills have accumulated or eroded, cannot attribute the gap to AI use alone, and does not provide a directly validated measure of cognitive debt itself. Building on these results, we propose the AI-Assisted Task–Feynman Technique Cognitive Debt Framework, a preliminary diagnostic and monitoring tool based on these performance-gap indicators, and discuss implications for the quality of AI-integrated programming education (SDG 4).

Discover Sustainability
Begum Rokeya University (BD), University of Dhaka (BD)
Teaching and Learning Programming
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon