Cost-Sensitive and Leakage-Aware Evaluation of Credit Card Fraud Detection Under Extreme Class Imbalance

Credit-card fraud detection is difficult because fraudulent transactions represent a very small share of observed payments, while false negatives and false positives impose different operational costs. This study presents a reproducible framework for evaluating fraud-detection classifiers under extreme class imbalance, with explicit controls for data leakage, threshold selection, and decision cost. The protocol covers dataset validation, partition design, training-only preprocessing and resampling, baseline models, validation-based threshold selection, and reporting of ranking, classification, calibration, and workload measures. It prioritizes average precision, precision, recall, false-positive rate, confusion counts, and alert volume rather than accuracy alone. The public European Credit Card Fraud Detection benchmark is described as containing 284,807 transactions, including 492 fraud cases, for a fraud prevalence of approximately 0.1727%. From these published aggregate counts, an all-legitimate rule achieves approximately 99.8273% accuracy while detecting no fraud. This analytical baseline demonstrates why accuracy alone is inadequate. The transaction-level CSV was not available in the execution environment used for this study; consequently, no trained-model comparison or model-specific performance claim is presented. The contribution is a transparent evaluation design and a verified calculation from published aggregate counts, not a ranking of fitted classifiers. The paper also examines temporal validity, anonymized features, privacy, fairness, calibration, and the boundary between benchmark results and production readiness. Keywords: Credit-card fraud detection; extreme class imbalance; data leakage; cost-sensitive evaluation; decision thresholds; precision–recall analysis.