Do Sequence Deep Learning Models Help Healthcare Provider Fraud Detection? A Two-Dataset, Multi-Split Benchmark of an Attention- Enhanced CNN-RNN against Tree Ensembles and Tabular Deep Learning

Abstract Attention-based sequence models are increasingly proposed for healthcare fraud detection but are rarely compared with strong tabular baselines. We test whether an attention-enhanced CNN-RNN helps on Medicare provider data and whether its attention weights indicate feature importance. We compared nine models, from an LR-RF hybrid and gradient-boosted trees to an MLP, FT-Transformer, CNN-RNN and attention CNN-RNN, over repeated random splits with natural-distribution test sets and validation-tuned thresholds. Dataset A has 100,000 Medicare provider records labelled by a statistical rule on submitted charges; we evaluated it with and without the label-defining charge features. Dataset B has 5,410 providers with fraud labels supplied with the data. We also retrained the attention model with shuffled feature order. On dataset A, gradient-boosted trees achieved a fraud F1 of 0.952. This is much better than the best neural model (0.769) and the attention CNN-RNN (0.750). When charge features were removed, every model’s performance dropped sharply, with scores falling between 0.42 and 0.52. On dataset B, all nine models scored between 0.588 and 0.629. This variation is similar to what is seen between train-test splits. Attention never reliably improved the CNN-RNN, and its weights followed input position rather than features. The large advantage of tree ensembles on rule-labelled data reflects reconstruction of the labelling rule; with labels supplied with the data, architecture choice made little difference. Sequence-based models and attention did not provide reliable improvements on either dataset. Also, attention weights should not be considered as explanations for tabular fraud models.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-16
DOI
https://doi.org/10.21203/rs.3.rs-11365365/v1
Primary Topic
Imbalanced Data Classification Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Do Sequence Deep Learning Models Help Healthcare Provider Fraud Detection? A Two-Dataset, Multi-Split Benchmark of an Attention- Enhanced CNN-RNN against Tree Ensembles and Tabular Deep Learning

Wahab Khan, Zia Ullah
1 links
Research Square
Imbalanced Data Classification Techniques
preprint

Do Sequence Deep Learning Models Help Healthcare Provider Fraud Detection? A Two-Dataset, Multi-Split Benchmark of an Attention- Enhanced CNN-RNN against Tree Ensembles and Tabular Deep Learning

Wahab Khan, Zia Ullah
preprint en

Abstract

Abstract Attention-based sequence models are increasingly proposed for healthcare fraud detection but are rarely compared with strong tabular baselines. We test whether an attention-enhanced CNN-RNN helps on Medicare provider data and whether its attention weights indicate feature importance. We compared nine models, from an LR-RF hybrid and gradient-boosted trees to an MLP, FT-Transformer, CNN-RNN and attention CNN-RNN, over repeated random splits with natural-distribution test sets and validation-tuned thresholds. Dataset A has 100,000 Medicare provider records labelled by a statistical rule on submitted charges; we evaluated it with and without the label-defining charge features. Dataset B has 5,410 providers with fraud labels supplied with the data. We also retrained the attention model with shuffled feature order. On dataset A, gradient-boosted trees achieved a fraud F1 of 0.952. This is much better than the best neural model (0.769) and the attention CNN-RNN (0.750). When charge features were removed, every model’s performance dropped sharply, with scores falling between 0.42 and 0.52. On dataset B, all nine models scored between 0.588 and 0.629. This variation is similar to what is seen between train-test splits. Attention never reliably improved the CNN-RNN, and its weights followed input position rather than features. The large advantage of tree ensembles on rule-labelled data reflects reconstruction of the labelling rule; with labels supplied with the data, architecture choice made little difference. Sequence-based models and attention did not provide reliable improvements on either dataset. Also, attention weights should not be considered as explanations for tabular fraud models.

Research Square
City University of Science and Information Technology (PK)
Good health and well-being
Imbalanced Data Classification Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon