Data-driven Natural Language Processing for IT Incident Classification : A Comparative Analysis of Machine Learning Models

In enterprise IT environments, the accuracy and timeliness of incident classification are important to maintain operational efficiency and minimize service disruption. Manual ticket handling often leads to failures and inconsistencies due to human error. The study presents machine learning models -Logistic Regression, Random Forest, and XGBoost - for the automation of IT incident ticket classification based on natural language description. Using a dataset of 47,837 real-world incidents, TF-IDF was used for feature extraction, and the models were evaluated using accuracy, F1-score, and confusion matrices. As a step towards addressing some of the interpretability concerns of enterprise AI, Local Interpretable Model-Agnostic Explanations (LIME) are integrated to analyze the model's decisions at the individual prediction level. The study highlights the trade-offs between classification performance concerning explainability. While complex ensemble methods achieve predictive accuracy, they are not as transparent as simple models. The findings provide insights into how organizations can select and operationalize explainable NLP models in IT service management contexts.