The Amplifier Effect: Why Guided ChatGPT Predicts Academic Achievement

Abstract The integration of Large Language Models (LLMs) in higher education presents significant pedagogical challenges. While research has explored various applications, a gap exists in understanding their impact on high-level cognitive tasks like research idea formulation, and how students' pre-existing skills influence outcomes. This study employed a quasi-experimental design with 207 engineering undergraduates, divided into a control group, an unguided ChatGPT group, and a guided ChatGPT group. A binary classification pipeline was developed to predict high-quality research proposals. The optimal model, a Logistic Regression classifier, achieved 93% accuracy, demonstrating high predictive power. The results revealed a critical 'Amplifier Effect.' Feature importance analysis from the model demonstrated that the most significant predictor of achievement was not the AI tool itself, but a student's initial skill, specifically Specific_objective_pretest. The guided use of ChatGPT (Groups_3) acted as a powerful amplifier for this competence, emerging as another top positive predictor, while being in the control group was a strong negative predictor. The findings conclude that the pedagogical framework is more critical than the AI tool itself, suggesting its primary value is to amplify, not supplant, foundational student competence. This study provides empirical evidence for designing structured AI interventions that enhance student skills.