Do Language Models Know Which Drug Is Unsafe? Three Measurement Pitfalls and a Severity Gradient
Medication-safety benchmarks report that language models often choose a drug the patient's risk rules out, but not whether the models lack the knowledge or fail to use it. Separating the two for open models (7B–72B), with a decision rule written before each experiment, we find that three common measurement choices can each mislead. The answer key: on 25 random RxSafeBench interaction cases, three auditor models from different providers name the key's unsafe drug in only 20–28%. Drug popularity: on drug pairs drawn from FDA labels, how often a drug is cited separates interacting from non-interacting pairs (AUROC 0.79) better than any open model. Answer format: moving a model between written probabilities and one-token scoring shifts its accuracy by up to fifteen points. Once these are controlled, the knowledge is graded by severity, also within popularity strata. In a pre-registered confirmation on 900 new pairs, every model separates major interactions better than moderate ones; open models reach AUROC 0.59–0.73 on major and 0.45–0.57 on moderate interactions, while a frontier commercial model reaches 0.95 and 0.72, so the gap does not shrink where errors are most dangerous. Use is distractible: shown the risk sentence for the unsafe drug only, 8–72B models gain 14–24 points; when every candidate carries a similar-looking warning, as real labels do, they gain 1–9 points. A simple retriever and a hazard-review prompt make open models more cautious, not more accurate. Code, pre-registrations and raw model outputs are included.
Authors
- Sebastian Goin del Rio (ORCID: https://orcid.org/0009-0006-1147-9995)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24355134
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint