Evaluating ML Models for Code Smell Detection Using Multi-Criteria Analysis

Code smells degrade code quality and hinder maintainability. Research shows Machine Learning (ML) techniques effectively detect them. Evaluating ML models on a single metric to determine the best ML model is misleading, as it may overlook other crucial aspects of model performance. To address this issue, this study presents a robust Multi-Criteria Decision-Making (MCDM) framework for recommending the most suitable ML models by integrating multiple diverse performance metrics. Our approach assesses ML models based on accuracy, precision, recall, F1-score, MCC, AUC, balanced accuracy, and execution time to balance predictive performance with computational efficiency. The proposed framework has been validated using four industry-relevant datasets under two different scenarios: (a) considering all specified performance metrics, and (b) considering all specified performance metrics except execution time, demonstrating its practical effectiveness and real-world applicability. The findings suggest that execution time plays a critical role in model selection, particularly useful in industry settings where efficiency is paramount. Statistical analysis using the Wilcoxon Signed-Rank Test confirms a significant trade-off between execution time and model performance, emphasizing computational efficiency’s impact on predictive accuracy. This study not only enhances model selection for code smell detection but also lays the groundwork for future research in optimizing ML models for practical deployment in software engineering workflows. Thus, the proposed framework offers a structured way to compare ML models with a balanced evaluation across multiple performance metrics.