A Text-Based Project Risk Classification System Using Multi-Model AI: Comparing SVM, Logistic Regression, Random Forests, Naive Bayes, and XGBoost

This study presents the design and evaluation of a multi-model artificial intelligence (AI) framework for proactive quality risk management in projects. A dataset comprising 2000 risk records was developed, containing four columns: Risk Description (input), Risk Category, Trigger, and Impact (outputs). Each output variable was modeled using three independent classifiers, forming a multi-step decision-making pipeline where one input is processed by multiple specialized models. Two feature extraction techniques, Term Frequency–Inverse Document Frequency (TF-IDF) and GloVe100 Word Embeddings, were compared in combination with several machine learning algorithms, including Logistic Regression, Support Vector Machines (SVMs), Random Forest, Multinomial Naive Bayes, and XGBoost. Results showed that model performance varied with task complexity and the number of output classes. Trigger prediction (28 classes), Logistic Regression, and SVM achieved the best performance, with a macro-average F1-score of 0.75, while XGBoost with TF-IDF features produced the highest accuracy for Risk Category classification (five classes). In Impact prediction (15 classes), SVM with Word Embeddings demonstrated superior results. The implementation, conducted in Python (v3.9.12, Anaconda), utilized Scikit-learn, XGBoost, SHAP, and Gensim libraries. SHAP visualizations and confusion matrices enhanced model interpretability. The proposed framework contributes to scalable, text-based, predictive, quality risk management, supporting real-time project decision-making.