From plausible to contestable: an audit framework for qualitative interpretation in AI-assisted health research

Abstract Background Qualitative analyses are usually appraised at study level; from a published report, an interpretive claim can rarely be traced to the excerpt that supports it. When a large language model (LLM), cloud-based or local, takes part in interpretation, a further question arises: what the model was given, what it proposed, and what the researcher kept. Reporting guidance for LLM research documents models, prompts and settings, not this excerpt-level pathway. Methods We decomposed excerpt-level interpretation into analytic operations and derived, for each, what must be recorded or verified when an LLM takes part. An automated run applied the framework with a local open-weight model required by transcript confidentiality to 206 excerpts from 18 interviews on territorial health inequalities, checking production, storage and routing of records. Context reduction, trajectory processing and human editing were not exercised. With status hidden, the author read 26 available repaired quotations mixed with 16 exact controls. Results The framework has seven components: an explicit analytic question; a declared status for every model input; a fixed limit on the context sent; units of meaning; deterministic verification of every quotation the model proposes; a closed catalogue of named warnings; and supervision records that never treat an unchanged proposal as accepted. Of 835 units, 795 (95.2%) quotations were found verbatim or after normalisation, 28 (3.4%) only after a discontinuous proposal was reduced to one segment, and 12 (1.4%) remained unlocated in the canonical transcript and were excluded from citable outputs. The retained segment fully supported the interpretation for 16 of 26 repaired and 14 of 16 exact quotations; we recommend researcher confirmation before citing repaired quotations. The review queue held 164 of 206 excerpts. The automated counts concern quotation location and record production; they do not verify interpretations, syntheses or contextual attributions. Conclusions The framework specifies what any model’s contribution to qualitative interpretation should leave behind for it to be inspected and contested. This assessment shows that these records can be produced at corpus scale in one configuration; whether they make interpretations more inspectable for others, and how their warnings should be calibrated, remains to be tested.

Authors

Publication Details

Journal
BMC Medical Research Methodology
Published
2026-11-16
DOI
https://doi.org/10.1186/s12874-026-03084-9
Primary Topic
Qualitative Research Methods and Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

From plausible to contestable: an audit framework for qualitative interpretation in AI-assisted health research

Nicolas Naïditch
1 links
BMC Medical Research Methodology
Qualitative Research Methods and Applications
article

From plausible to contestable: an audit framework for qualitative interpretation in AI-assisted health research

Nicolas Naïditch
article en

Abstract

Abstract Background Qualitative analyses are usually appraised at study level; from a published report, an interpretive claim can rarely be traced to the excerpt that supports it. When a large language model (LLM), cloud-based or local, takes part in interpretation, a further question arises: what the model was given, what it proposed, and what the researcher kept. Reporting guidance for LLM research documents models, prompts and settings, not this excerpt-level pathway. Methods We decomposed excerpt-level interpretation into analytic operations and derived, for each, what must be recorded or verified when an LLM takes part. An automated run applied the framework with a local open-weight model required by transcript confidentiality to 206 excerpts from 18 interviews on territorial health inequalities, checking production, storage and routing of records. Context reduction, trajectory processing and human editing were not exercised. With status hidden, the author read 26 available repaired quotations mixed with 16 exact controls. Results The framework has seven components: an explicit analytic question; a declared status for every model input; a fixed limit on the context sent; units of meaning; deterministic verification of every quotation the model proposes; a closed catalogue of named warnings; and supervision records that never treat an unchanged proposal as accepted. Of 835 units, 795 (95.2%) quotations were found verbatim or after normalisation, 28 (3.4%) only after a discontinuous proposal was reduced to one segment, and 12 (1.4%) remained unlocated in the canonical transcript and were excluded from citable outputs. The retained segment fully supported the interpretation for 16 of 26 repaired and 14 of 16 exact quotations; we recommend researcher confirmation before citing repaired quotations. The review queue held 164 of 206 excerpts. The automated counts concern quotation location and record production; they do not verify interpretations, syntheses or contextual attributions. Conclusions The framework specifies what any model’s contribution to qualitative interpretation should leave behind for it to be inspected and contested. This assessment shows that these records can be produced at corpus scale in one configuration; whether they make interpretations more inspectable for others, and how their warnings should be calibrated, remains to be tested.

BMC Medical Research Methodology
Qualitative Research Methods and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon