A framework for establishing prima facie validity and quality in scoping review evidence maps

Scoping reviews map the breadth, characteristics, and distribution of research activity across a field, typically without the formal critical appraisal required of systematic reviews. Implicit in this approach is the assumption that “the literature” is a coherent and credible representation of the underlying science. In a publishing ecosystem marked by heterogeneous editorial standards, variable peer review, and expanding volumes of low-quality or poorly governed research, this assumption is increasingly untenable and may lead scoping reviews to overstate the validity of the evidence they represent. We conceptually interrogated the boundaries of “the literature” as an evidentiary construct and examined whether scoping reviews, as currently operationalized, implicitly overstate scientific validity. Drawing on existing literature concerning predatory publishing, bibliometric indicators, authorship behavior, and research integrity, we developed a structured yet proportionate framework of descriptive quality indicators that can be extracted alongside standard scoping review data items without imposing exclusion criteria or formal appraisal. The framework comprises three domains. Domain 1) Author Concentration and Publication Patterns—documents authorship concentration, publication velocity, self-citation, redundancy (“salami slicing”), and scholarly integrity signals such as retractions and corrections. Domain 2) Journal Provenance and Indexing Status—characterizes indexing category and stability, delisting, publication model, and journal-level integrity signals. Domain 3) Data Trustworthiness—captures six markers: ethics approval, prospective or retrospective trial registration, use of reporting standards, data collection dates, handling of retractions and corrections, and editorial responses. A proposed checklist operationalizes these indicators as simple extraction fields. The three domains provide a descriptive framework that enhances the interpretability of scoping review evidence maps by illuminating the conditions under which the mapped evidence was produced, disseminated, and maintained. Rather than excluding studies, the approach contextualizes credibility, strengthens transparency, and supports more informed downstream decision-making in research planning, policy development, and systematic review design. Integrating these contextual markers helps scoping reviews accurately represent both the contours and the credibility of the scientific landscapes they describe.

Authors

Institutions

Publication Details

Journal
Research Integrity and Peer Review
Published
2026-11-13
DOI
https://doi.org/10.1186/s41073-026-00268-1
Primary Topic
Meta-analysis and systematic reviews
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

A framework for establishing prima facie validity and quality in scoping review evidence maps

Tyler Davis-Sandfoss, Michelle S. High, Marsha J. Bennett, Karen A. Domínguez-Cancino et al.
3 links
Research Integrity and Peer Review
Meta-analysis and systematic reviews
article

A framework for establishing prima facie validity and quality in scoping review evidence maps

Tyler Davis-Sandfoss, Michelle S. High, Marsha J. Bennett, Karen A. Domínguez-Cancino, Kelly Mecham, Audrey Rosenblatt, Patrick Palmieri, Catherine Chung
article en

Abstract

Scoping reviews map the breadth, characteristics, and distribution of research activity across a field, typically without the formal critical appraisal required of systematic reviews. Implicit in this approach is the assumption that “the literature” is a coherent and credible representation of the underlying science. In a publishing ecosystem marked by heterogeneous editorial standards, variable peer review, and expanding volumes of low-quality or poorly governed research, this assumption is increasingly untenable and may lead scoping reviews to overstate the validity of the evidence they represent. We conceptually interrogated the boundaries of “the literature” as an evidentiary construct and examined whether scoping reviews, as currently operationalized, implicitly overstate scientific validity. Drawing on existing literature concerning predatory publishing, bibliometric indicators, authorship behavior, and research integrity, we developed a structured yet proportionate framework of descriptive quality indicators that can be extracted alongside standard scoping review data items without imposing exclusion criteria or formal appraisal. The framework comprises three domains. Domain 1) Author Concentration and Publication Patterns—documents authorship concentration, publication velocity, self-citation, redundancy (“salami slicing”), and scholarly integrity signals such as retractions and corrections. Domain 2) Journal Provenance and Indexing Status—characterizes indexing category and stability, delisting, publication model, and journal-level integrity signals. Domain 3) Data Trustworthiness—captures six markers: ethics approval, prospective or retrospective trial registration, use of reporting standards, data collection dates, handling of retractions and corrections, and editorial responses. A proposed checklist operationalizes these indicators as simple extraction fields. The three domains provide a descriptive framework that enhances the interpretability of scoping review evidence maps by illuminating the conditions under which the mapped evidence was produced, disseminated, and maintained. Rather than excluding studies, the approach contextualizes credibility, strengthens transparency, and supports more informed downstream decision-making in research planning, policy development, and systematic review design. Integrating these contextual markers helps scoping reviews accurately represent both the contours and the credibility of the scientific landscapes they describe.

Research Integrity and Peer Review
Northwestern University (US), Lurie Children's Hospital (US), Universidad San Sebastián (CL), Universidad Peruana Unión (PE), Institute for Research and Theory Methodologies (US), Rush University (US), Touro University Nevada (US), Louisiana State University Health Sciences Center New Orleans (US)
Meta-analysis and systematic reviews
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon