Evaluating ChatGPT-driven Automated Test Generation for Personalized Programming Education

Large Language Models (LLMs), such as ChatGPT, hold immense potential to irrevocably influence the dimension of educational technology by providing empowerment for personalized learning experiences. This paper covers the integration of ChatGPT into existing eLearning platforms toward supporting Java programming education. Using artificial intelligence-based capabilities of ChatGPT in natural language processing, our software enables instructors to generate individual module assessments tailored to a student's profile with unprecedented ease. Evaluation of the effectiveness and efficiency of the system was performed by a comprehensive review of the system, obtaining feedback from instructors and students, and analyzing test performance metrics. The evaluation showed that most teachers found the system very user friendly, with significant savings in time for test creation. Satisfaction related to personalized tests designed by ChatGPT was also adequate, and the average scores achieved on test cases set by ChatGPT were relatively high compared to those manually curated. Results underline the potential of ChatGPT-driven automated test generation for enhancing personalized programming education on eLearning platforms, making available tailored assessment, by consideration of individual needs of students, and increased learning outcomes and efficiency.