Identifying Interpretable Features Impacting Nontraditional Undergraduate Computer Science Student Retention

This research full paper describes an investigation of nontraditional undergraduate Computer Science (CS) students' unique retention issues by using machine learning. It presents new insight into the most critical factors impacting their retention compared to traditional students. CS student retention is a pressing concern for many universities. Traditional students have been well-studied, but the growing number of nontraditional students have received less attention. These students often face unique challenges due to their different backgrounds and experiences compared to their traditional counterparts. For example, they typically delay postsecondary enrollment, have dependents, and attend school part time. Existing research has limited information on how these varying characteristics influence their academic success. Consequently, applying retention strategies based solely on traditional student data might not be effective for nontra-ditional students. This study aims to bridge the knowledge gap regarding nontraditional undergraduate CS student retention. We proposed two key questions: (1) What are the primary factors influencing nontraditional undergraduate CS student attrition? and (2) How can we predicate students at a higher risk of dropping out at an earlier stage? We analyzed the data of nontraditional undergraduate students enrolled in the CS major at a university with a predominantly nontraditional student population. The data span 13 years and encompass demographic, academic, and behavioral features. We trained five machine learning models (Logistic Regression, Support Vector Machines, K-Nearest Neighbors, Random Forest, and Gradient Boosting Trees) and an ensemble model. By considering both accuracy and AUC (Area Under the Curve) of the optimized models, we found that overall, the Gradient Boosting Trees model performs the best. Using permutation importance in the Gradient Boosting Trees model, we found nine most significant features impacting student retention: (1) GPA. Students with higher GPA (cumulative or in the last semester) are more likely to retain. (2) Course load. Students taking more classes overall or in a given semester tend to retain better. (3) Number of face-to-face classes taken. Students taking fewer face-to-face classes and more online classes are more likely to retain. (4) Age. Younger students tend to retain better. (5) Whether students transferred through a community college partnership. Students recruited through a partnership with two-year institutions tend to retain better. (6) Financial aid. Students with financial aid are less likely to drop out than students without any financial aid. These innovative findings deepen the knowledge of the specific retention challenges faced by nontraditional undergraduate CS students. This study successfully addressed the proposed research questions, providing valuable insights into nontraditional undergrad-uate CS student retention. Identifying key influential features improves the interpretability of the predictive models, leading to better compliance with regulations and reduced concerns about bias. Additionally, the optimized model can inform interventions and student services to enhance retention rates.