The use and methodological reporting of large language models in qualitative research: a scoping review

BACKGROUND: Large language models (LLMs) are being integrated into qualitative research processes, yet the scope, function, and reporting quality of their use remain poorly understood. Existing reporting guidelines for qualitative research, including for example the Consolidated Criteria for Reporting Qualitative Research (COREQ), provide minimal guidance for documenting LLM use. This scoping review provides an overview of the emerging use of LLMs applications in qualitative research and assesses the associated reporting practices. METHODS: A scoping review was conducted following the PRISMA-ScR guidelines and the Joanna Briggs Institute methodological framework. Five databases (PubMed, CINAHL, PsycINFO, Business Source Premier, and Scopus) were searched for peer-reviewed empirical studies published between January 2020 and May 2025 that employed at least one LLM in a substantive qualitative research stage. The search yielded 5,049 records, of which 4,201 remained after duplicate removal. Studies were screened independently by multiple reviewers, and data were extracted using a standardized template capturing study metadata, methodological characteristics, and comprehensive LLM implementation details. RESULTS: Seventy-five studies were included. OpenAI GPT models dominated the field, appearing in 93% of studies. LLMs were applied across the full spectrum of qualitative research, with coding assistance (n = 43) and theme identification (n = 41) as the most common applications. Thematic analysis was the predominant qualitative method (n = 38), and content analysis (n = 12). Technical reporting was highly inconsistent: only 13 studies reported temperature settings, 12 documented context length, and 4 provided top_p values. Approximately half of studies (45%, n = 34) did not specify the deployment configuration (API, web interface, or local), and 75% (n = 56) reported no parameter settings at all. While 61% of studies provided complete or partial prompts, 13% reported no prompting details. Agreement rates between LLM and human coders ranged from 36% to 99%, reflecting substantial variation related to task complexity, prompt engineering quality, and validation rigor. Nearly all studies (95%) discussed ethical considerations, and 97% incorporated human verification of AI outputs. DISCUSSION: LLMs have been adopted across qualitative research workflows, yet critical methodological details are frequently underreported, undermining comparability. The findings highlight an urgent need for dedicated reporting guidelines, such as the COREQ + LLM extension, to ensure that LLM-assisted qualitative research meets standards of transparency, rigor, and interpretive depth. Future research should address the predominance of proprietary models, the limited evidence for non-English contexts, and the need for systematic comparison of models, prompting strategies, and validation approaches.

Authors

Institutions

Publication Details

Journal
BMC Medical Research Methodology
Published
2026-06-16
DOI
https://doi.org/10.1186/s12874-026-02913-1
Primary Topic
Computational and Text Analysis Methods
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

The use and methodological reporting of large language models in qualitative research: a scoping review

Leonard Fehring, Christian Kempny, Paul Rust, Sven Meister et al.
BMC Medical Research Methodology
Computational and Text Analysis Methods
article

The use and methodological reporting of large language models in qualitative research: a scoping review

Leonard Fehring, Christian Kempny, Paul Rust, Sven Meister, Julian Frings
article en

Abstract

BACKGROUND: Large language models (LLMs) are being integrated into qualitative research processes, yet the scope, function, and reporting quality of their use remain poorly understood. Existing reporting guidelines for qualitative research, including for example the Consolidated Criteria for Reporting Qualitative Research (COREQ), provide minimal guidance for documenting LLM use. This scoping review provides an overview of the emerging use of LLMs applications in qualitative research and assesses the associated reporting practices. METHODS: A scoping review was conducted following the PRISMA-ScR guidelines and the Joanna Briggs Institute methodological framework. Five databases (PubMed, CINAHL, PsycINFO, Business Source Premier, and Scopus) were searched for peer-reviewed empirical studies published between January 2020 and May 2025 that employed at least one LLM in a substantive qualitative research stage. The search yielded 5,049 records, of which 4,201 remained after duplicate removal. Studies were screened independently by multiple reviewers, and data were extracted using a standardized template capturing study metadata, methodological characteristics, and comprehensive LLM implementation details. RESULTS: Seventy-five studies were included. OpenAI GPT models dominated the field, appearing in 93% of studies. LLMs were applied across the full spectrum of qualitative research, with coding assistance (n = 43) and theme identification (n = 41) as the most common applications. Thematic analysis was the predominant qualitative method (n = 38), and content analysis (n = 12). Technical reporting was highly inconsistent: only 13 studies reported temperature settings, 12 documented context length, and 4 provided top_p values. Approximately half of studies (45%, n = 34) did not specify the deployment configuration (API, web interface, or local), and 75% (n = 56) reported no parameter settings at all. While 61% of studies provided complete or partial prompts, 13% reported no prompting details. Agreement rates between LLM and human coders ranged from 36% to 99%, reflecting substantial variation related to task complexity, prompt engineering quality, and validation rigor. Nearly all studies (95%) discussed ethical considerations, and 97% incorporated human verification of AI outputs. DISCUSSION: LLMs have been adopted across qualitative research workflows, yet critical methodological details are frequently underreported, undermining comparability. The findings highlight an urgent need for dedicated reporting guidelines, such as the COREQ + LLM extension, to ensure that LLM-assisted qualitative research meets standards of transparency, rigor, and interpretive depth. Future research should address the predominance of proprietary models, the limited evidence for non-English contexts, and the need for systematic comparison of models, prompting strategies, and validation approaches.

BMC Medical Research MethodologyVol. 26(1)
Witten/Herdecke University (DE), Helios Universitätsklinikum Wuppertal (DE), Fraunhofer Institute for Software and Systems Engineering (DE)
Openalex Percentile: Top 3%
Computational and Text Analysis Methods
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.