ArtefactSense: Do LLMs Notice When a Wearable Biosignal Is Unreadable?

Language models are now asked to read raw wearable signals, yet real recordings can be corrupted by motion, lost skin contact, amplifier saturation or mains hum. A trustworthy reader should then flag the window instead of answering. ArtefactSense, the dataset, generator, scorer and harness we release, injects graded artefacts into real ECG, PPG and EMG windows and asks a question whose ground truth and usability label come from a reference pipeline; the frozen release holds 1500 items, 21.8% of them unusable, read by Claude and open-weight models as text and images. AURC, the area under the error-rate curve as more items are answered (lower is better), is 0.125–0.282 for the Claude models and 0.565–0.847 for open-weight ones, yet no model abstains on more than 23.5% (Claude) or 58.7% (open-weight) of unusable items. Claude's stated confidence on unusable items piles at 0.5, so the share it answers confidently is 36.7–65.1% at a 0.5 cut-off and 0.0–12.5% at a 0.7 cut-off. A non-LLM baseline of classic signal-quality rules, fit leave-one-subject-out on a separate tuning set, flags 85.9% of unusable items, and gating a Claude model with it leaves at most 10.1% unflagged. Under matched generation settings Haiku and Sonnet still leave 52.4–62.7% of unusable items unflagged, and a quality-first instruction raises that share in 5 of 6 rows. Dataset, code and scorer are released at https://doi.org/10.5281/zenodo.24185517.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-13
DOI
https://doi.org/10.5281/zenodo.24150268
Citations
1
Primary Topic
ECG Monitoring and Analysis
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

ArtefactSense: Do LLMs Notice When a Wearable Biosignal Is Unreadable?

Tharun Reddy Kandukuri, Giorgio Mallia
1 citations
Zenodo (CERN European Organization for Nuclear Research)
ECG Monitoring and Analysis
preprint

ArtefactSense: Do LLMs Notice When a Wearable Biosignal Is Unreadable?

Tharun Reddy Kandukuri, Giorgio Mallia
preprint en
1 citations

Abstract

Language models are now asked to read raw wearable signals, yet real recordings can be corrupted by motion, lost skin contact, amplifier saturation or mains hum. A trustworthy reader should then flag the window instead of answering. ArtefactSense, the dataset, generator, scorer and harness we release, injects graded artefacts into real ECG, PPG and EMG windows and asks a question whose ground truth and usability label come from a reference pipeline; the frozen release holds 1500 items, 21.8% of them unusable, read by Claude and open-weight models as text and images. AURC, the area under the error-rate curve as more items are answered (lower is better), is 0.125–0.282 for the Claude models and 0.565–0.847 for open-weight ones, yet no model abstains on more than 23.5% (Claude) or 58.7% (open-weight) of unusable items. Claude's stated confidence on unusable items piles at 0.5, so the share it answers confidently is 36.7–65.1% at a 0.5 cut-off and 0.0–12.5% at a 0.7 cut-off. A non-LLM baseline of classic signal-quality rules, fit leave-one-subject-out on a separate tuning set, flags 85.9% of unusable items, and gating a Claude model with it leaves at most 10.1% unflagged. Under matched generation settings Haiku and Sonnet still leave 52.4–62.7% of unusable items unflagged, and a quality-first instruction raises that share in 5 of 6 rows. Dataset, code and scorer are released at https://doi.org/10.5281/zenodo.24185517.

Zenodo (CERN European Organization for Nuclear Research)
ECG Monitoring and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon