Reassessing Unsupervised Anomaly Augmentation for Fraud Detection: The Roles of Distribution Shift, Learner Instability, and Run-to-Run Variability

Unsupervised anomaly scores are often added to supervised fraud classifiers, but an apparent gain may stem from learner instability or from the mere addition of a column rather than from anomaly information. We propose a controlled evaluation protocol that combines temporal splitting, multi-seed replication, uninformative negative-control columns, learner-stability checks and prespecified equivalence margins. It is applied to LightGBM and XGBoost with six anomaly representations on the Bank Account Fraud (BAF) benchmark and the chronologically split CreditCard dataset, including rolling-origin windows. On BAF, a single-seed gain of 3.5% in mean-variant AUPRC shrank to 1.2% under multi-seed evaluation, matching the 1.1% gain of a random-noise column. On CreditCard, changing only the LightGBM bagging seed moved baseline AUPRC between 0.004 and 0.742. Bounding leaf values reduced collapses from 77% of runs to none, and a reverse experiment reproduced the collapses in XGBoost once both of its leaf-value safeguards were removed. With stable learners, every anomaly representation was equivalent to a random-noise column within prespecified margins of about 1% (CreditCard) and 4% (BAF) of baseline AUPRC. Negative controls, multi-seed evaluation and learner-stability checks should precede any attribution of a gain to anomaly information.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-11-19
DOI
https://doi.org/10.3390/app162211473
Primary Topic
Imbalanced Data Classification Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Reassessing Unsupervised Anomaly Augmentation for Fraud Detection: The Roles of Distribution Shift, Learner Instability, and Run-to-Run Variability

Canan Aslanyürek, Murat Aslanyürek
Applied Sciences
Imbalanced Data Classification Techniques
article

Reassessing Unsupervised Anomaly Augmentation for Fraud Detection: The Roles of Distribution Shift, Learner Instability, and Run-to-Run Variability

Canan Aslanyürek, Murat Aslanyürek
article en

Abstract

Unsupervised anomaly scores are often added to supervised fraud classifiers, but an apparent gain may stem from learner instability or from the mere addition of a column rather than from anomaly information. We propose a controlled evaluation protocol that combines temporal splitting, multi-seed replication, uninformative negative-control columns, learner-stability checks and prespecified equivalence margins. It is applied to LightGBM and XGBoost with six anomaly representations on the Bank Account Fraud (BAF) benchmark and the chronologically split CreditCard dataset, including rolling-origin windows. On BAF, a single-seed gain of 3.5% in mean-variant AUPRC shrank to 1.2% under multi-seed evaluation, matching the 1.1% gain of a random-noise column. On CreditCard, changing only the LightGBM bagging seed moved baseline AUPRC between 0.004 and 0.742. Bounding leaf values reduced collapses from 77% of runs to none, and a reverse experiment reproduced the collapses in XGBoost once both of its leaf-value safeguards were removed. With stable learners, every anomaly representation was equivalent to a random-noise column within prespecified margins of about 1% (CreditCard) and 4% (BAF) of baseline AUPRC. Negative controls, multi-seed evaluation and learner-stability checks should precede any attribution of a gain to anomaly information.

Applied SciencesVol. 16(22)
Kırklareli University (TR)
Imbalanced Data Classification Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon