Hyperparameter Tuning of Random Forest using Social Group Optimization Algorithm for Credit Card Fraud Detection in Banking Data

As the adoption of credit cards continues to expand alongside advancements in e-commerce, the frequency and complexity of fraudulent activities have also grown, posing significant challenges for the financial sector. Detecting fraudulent transactions within highly imbalanced datasets remains a critical issue in ensuring secure banking operations. This study explores a robust approach RF_SGO to credit card fraud detection by combining pre-processing techniques such as Synthetic Minority Oversampling Technique (SMOTE) and class weight adjustment with Random Forest (RF) models optimized using the Social Group Optimization (SGO) algorithm. Additionally, the study utilizes Random Forest's feature importance mechanism to identify the most influential features contributing to fraud detection, enhancing interpretability and decision-making. Our methodology evaluates RF_SGO across three datasets: the original European cardholders' imbalanced dataset, a class-weight-adjusted dataset, and a SMOTE-enhanced dataset. Model performance is measured using key metrics, including Accuracy, Precision, Recall, F1-Score, and ROC-AUC. The RF_SGO model demonstrated superior performance, with the SMOTE-enhanced variant achieving the highest ROC-AUC (0.98) and Recall (0.88), effectively balancing sensitivity and specificity. The class-weighted RF_SGO achieved the highest Precision (0.96), making it ideal for minimizing false positives. Furthermore, the feature importance analysis identified key predictors of fraudulent behavior, providing actionable insights for financial institutions. Comparisons with traditional machine learning algorithms (e.g., Logistic Regression, Decision Trees, and SVM) and advanced models (e.g., XGBoost, CatBoost, and deep learning) highlight RF_SGO's ability to outperform in precision-recall trade-offs and overall classification effectiveness. This study underscores the significance of incorporating hyperparameter tuning, feature importance analysis, and data balancing strategies to improve fraud detection. The proposed RF_SGO framework offers a scalable and efficient solution for financial institutions to mitigate fraud, ensuring more reliable and secure transaction systems.