Analysis of Generative AI Usage in Programming Courses: Performance and Failures

In the field of studying information technology, programming is one of the fundamental aspects. As college students live in the modern world, exposure to different tools that aid in achieving academic goals is common, including the usage of different AI chatbots such as ChatGPT. This paper explores the application of different machine learning models, particularly Bi-GRU, CNN, and Bi-LTSM in getting sentiments students from a university in National Capital Region, Philippines, who are taking programming classes regarding the usage of ChatGPT for educational purposes. The evaluation revealed that the CNN Model had the best accuracy of 0.94 compared to Bi-GRU with 0.90 and Bi-LSTM with 0.86. Though CNN had the highest accuracy, Bi-GRU was the most consistent model with a balanced score of 0.94 both in training and validation. These are measures of a model's risk of overfitting and its capability to generalize well. Exploratory Data Analysis via N-grams and Wordcloud was conducted on the predicted sentiments to get an overview of the characteristics of the positive and negative sentiments of the students.