Reassessing Unsupervised Anomaly Augmentation for Fraud Detection: The Roles of Distribution Shift, Learner Instability, and Run-to-Run Variability
Unsupervised anomaly scores are often added to supervised fraud classifiers, but an apparent gain may stem from learner instability or from the mere addition of a column rather than from anomaly information. We propose a controlled evaluation protocol that combines temporal splitting, multi-seed replication, uninformative negative-control columns, learner-stability checks and prespecified equivalence margins. It is applied to LightGBM and XGBoost with six anomaly representations on the Bank Account Fraud (BAF) benchmark and the chronologically split CreditCard dataset, including rolling-origin windows. On BAF, a single-seed gain of 3.5% in mean-variant AUPRC shrank to 1.2% under multi-seed evaluation, matching the 1.1% gain of a random-noise column. On CreditCard, changing only the LightGBM bagging seed moved baseline AUPRC between 0.004 and 0.742. Bounding leaf values reduced collapses from 77% of runs to none, and a reverse experiment reproduced the collapses in XGBoost once both of its leaf-value safeguards were removed. With stable learners, every anomaly representation was equivalent to a random-noise column within prespecified margins of about 1% (CreditCard) and 4% (BAF) of baseline AUPRC. Negative controls, multi-seed evaluation and learner-stability checks should precede any attribution of a gain to anomaly information.
Authors
- Canan Aslanyürek (ORCID: https://orcid.org/0000-0001-7513-9953)
- Murat Aslanyürek (ORCID: https://orcid.org/0000-0002-3296-4395)
Institutions
- Kırklareli University (TR)
Publication Details
- Journal
- Applied Sciences
- Published
- 2026-11-19
- DOI
- https://doi.org/10.3390/app162211473
- Primary Topic
- Imbalanced Data Classification Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00