Adaptive Hybrid Learning for Credit Card Fraud Detection: A Comparative Study of Supervised, Reinforcement, and Hybrid Models

Financial fraud detection is challenged by the rarity and evolving nature of fraudulent transactions. This paper presents a comparative study of traditional classifiers (Logistic Regression, Random Forest, SGDClassifier), a reinforcement learning approach (Q-learning), and a novel hybrid model that combines Q-learning-driven feature selection with Random Forest classification. The hybrid framework applies SMOTE selectively to address class imbalance and leverages Bayesian optimization to fine-tune hyperparameters-striking a balance between computational efficiency and detection accuracy. To ensure the reliability of results, experiments were conducted on the Kaggle Credit Card Fraud Detection dataset-comprising 284,807 transactions with only <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{0. 1 7 \%}$</tex> labeled as fraudulent-and repeated across five independent runs, with average performance metrics reported. Among the traditional classification models, Random Forest demonstrated the highest precision (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.9847 \pm$</tex> 0.0109), while Logistic Regression delivered a strong balance between F1-score (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.9093 \pm 0.0160$</tex>) and ROC-AUC (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.9724 \pm 0.00)$</tex>. The Q-learning model, on the other hand, struggled with lower recall (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.7686 \pm 0.0049$</tex>), likely due to challenges posed by the class imbalance. The proposed hybrid approach, which integrates Q-learning-based feature selection with Random Forest classification, achieved an F1-score of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.8916 \pm 0.0894$</tex> and a ROCAUC of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.95 \pm 0.07$</tex>. This reflects a relative improvement of up to 5% in F1-score over standalone Q-learning and demonstrates competitive performance when compared to conventional models. Overall, the findings highlight the hybrid model's potential as a stable and scalable solution for detecting fraud in real time.