Understanding and Enhancing CS Students’ Interaction Experience with AI Coding Assistant Tools

AI coding assistants (ACATs) are reshaping computer science (CS) education, yet students’ perception and responses to ACATs’ suggestions remains limited understood, especially regarding behavioral patterns, decision-making, and usability challenges. To address this gap, we conducted a study with 27 CS students, examining their interactions with three widely used ACATs across five key dimensions: interaction frequency and acceptance rate, self-perceived productivity, behavioral patterns, decision-making factors, and challenges and expectations. To support this investigation, we developed an experimental platform incorporating a VSCode extension for log data collection, screen recording and automatic generation of personalized interview and survey questions. Our findings reveal substantial variation in ACAT acceptance rates depending on task types, recommendation methods, and content. We propose a novel five-layer interaction behavior model that captures different stages of user interaction. Notable insights include the problem-solving value of rejected AI suggestions, the inefficiencies introduced by modifying existing code that often lead to backtracking, and the high stability of “slowly accepted” suggestions. Moreover, we identify 22 decision-making factors, 11 challenges, and 23 student expectations for future ACAT improvements—such as enhanced debugging accuracy and adaptive learning of individual coding styles. This study contributes actionable design implications for improving ACAT usability, informing student interaction strategies, and guiding future research in human-software interaction, ultimately aiming to better support CS education.

Authors

Institutions

Publication Details

Journal
ACM Transactions on Software Engineering and Methodology
Published
2025-12-23
DOI
https://doi.org/10.1145/3785479
Primary Topic
Teaching and Learning Programming
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Understanding and Enhancing CS Students’ Interaction Experience with AI Coding Assistant Tools

Xiao Long, Li Zhang, Yinghao Zhu, Xin Tan et al.
ACM Transactions on Software Engineering and Methodology
Teaching and Learning Programming
article

Understanding and Enhancing CS Students’ Interaction Experience with AI Coding Assistant Tools

Xiao Long, Li Zhang, Yinghao Zhu, Xin Tan, Jing Jiang
article en

Abstract

AI coding assistants (ACATs) are reshaping computer science (CS) education, yet students’ perception and responses to ACATs’ suggestions remains limited understood, especially regarding behavioral patterns, decision-making, and usability challenges. To address this gap, we conducted a study with 27 CS students, examining their interactions with three widely used ACATs across five key dimensions: interaction frequency and acceptance rate, self-perceived productivity, behavioral patterns, decision-making factors, and challenges and expectations. To support this investigation, we developed an experimental platform incorporating a VSCode extension for log data collection, screen recording and automatic generation of personalized interview and survey questions. Our findings reveal substantial variation in ACAT acceptance rates depending on task types, recommendation methods, and content. We propose a novel five-layer interaction behavior model that captures different stages of user interaction. Notable insights include the problem-solving value of rejected AI suggestions, the inefficiencies introduced by modifying existing code that often lead to backtracking, and the high stability of “slowly accepted” suggestions. Moreover, we identify 22 decision-making factors, 11 challenges, and 23 student expectations for future ACAT improvements—such as enhanced debugging accuracy and adaptive learning of individual coding styles. This study contributes actionable design implications for improving ACAT usability, informing student interaction strategies, and guiding future research in human-software interaction, ultimately aiming to better support CS education.

ACM Transactions on Software Engineering and Methodology
Beijing Academy of Artificial Intelligence (CN), Beihang University (CN)
Openalex Percentile: Top 29%
Teaching and Learning Programming
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Understanding and Enhancing CS Students’ Interaction Experience with AI Coding Assistant Tools — Xiao Long, Li Zhang, et al. · ACM Transactions on Software Engineering and Methodology (2025) | TGRS Research Map | TGRS