An Evaluation of Low-Shot Learning Techniques for the Detection of Credit Card Fraud

The task of credit card fraud detection presents many obstacles to traditional machine learning techniques, most notably severe class imbalance. Meanwhile, Low-Shot Learning (LSL) techniques have shown great effectiveness in multi-class problems involving significant class rarity, but have seen little examination in binary tasks such as fraud detection. In this paper, we select two low-shot learning papers from the literature (representing two major approaches to LSL), and replicate their methods on a highly imbalanced credit-card fraud dataset, to compare their performances to that of six contemporary state-of-the-art (SOTA) models. In the process, we improve on the experimental methodologies presented in the two selected papers by introducing multiple runs of cross-fold validation, measurement of the imbalance-robust performance metric Area Under the Precision-Recall Curve (AUPRC), and statistical analyses of our results. To the best of our knowledge, our work is the first to compare the two different LSL approaches to non-LSL SOTA methods, as well as the first peer-reviewed paper to evaluate any LSL method on a fraud detection dataset. We find that, while our chosen optimization-based method, Meta-Balance, underperforms the SOTA baseline in terms of AUPRC, our similarity-based method, Siamese-RNN, significantly outperforms the SOTA, and yields the highest AUPRC score recorded in the literature on the Kaggle credit card fraud dataset.