The textual shell: language model evaluations in therapeutic drug monitoring test the numerical end and leave the interface unmeasured
Therapeutic drug monitoring (TDM) turns a measured concentration into a dose or a monitoring plan. It has a computational core and a textual shell: the record is read into inputs at the front, the number written into a report at the back. Large language models fail by fabrication and pharmacokinetic engines by specification, so the models belong on the shell. Reported performance spans 10% to 94.9% across incompatible scales, and this review asks whether the evaluations have looked where the models would work. Of thirty evaluations on the chain, twenty-eight test the model on work something else settles: a measured concentration once, a study-supplied criterion twenty-seven times. Where a measured concentration adjudicates, the comparator ran once — 37.0% against 15.6% within ±2 mg/L — and on the only shared denominator both methods sat near a third, where the dispersion of the data places them. At the back of the shell sit two studies: ten fictional cases scored by one pharmacologist, and eleven patient questions rated against expert answers. At the front, extraction from a TDM record is unevaluated in what we retrieved. Thirty-three evaluations across nine dimensions show why the shell is unmeasured: accuracy has an independent value in thirty-two, completeness in five, harm grading in ten, repeatability in seven, reference reliability in seven of twenty-five. Little headroom where the answer is measured; none where it is written. We set out what shell-side evaluation requires and propose a ten-item checklist.
Authors
- Yichao Xu (ORCID: https://orcid.org/0000-0002-0476-9787)
- Lei Zhang
Institutions
- Second Affiliated Hospital of Zhejiang University (CN)
- Zhejiang University (CN)
Publication Details
- Published
- 2026-11-07
- DOI
- https://doi.org/10.22541/authorea.15011835/v1
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint
Funders
- National Natural Science Foundation of China