From PubMed counts to evidence-review readiness: a query audit of named KEGG signaling pathways in extracellular-vesicle research

Abstract Background Extracellular vesicles (EVs) transfer proteins, lipids and nucleic acids between cells, with context-dependent effects. PubMed counts of named signaling-pathway searches are often read as evidence of where EV research is active or ready for synthesis, yet a reproducible query can retrieve records that mention a pathway without studying it. We tested whether named-pathway searches return the intended literature across EV research in cardiac repair, neurodegeneration, and tumor metastasis, and asked why some did not. Methods The primary universe was all 26 human maps in the KEGG Signal-transduction subcategory, plus a prespecified 13-map secondary layer. Before the definitive retrieval, raters judged up to ten records per map separately for pathway substance and EV/context relevance. Post hoc, reusing the ratings, we computed the unaudited ranking, tested whether Medical Subject Headings (MeSH) could have replaced the audit, and examined the recorded failure mechanisms. Results All 234 queries passed the technical rules, but only three primary maps yielded bounded source sets (Notch, NF-kappa B, mTOR). Ras was non-empty but audit-negative, three were empty, 18 required redesign, and one was a duplicate; the secondary layer reproduced the pattern. Across 139 audited units, 43.2% failed pathway substantiveness and 40.3% failed EV/context relevance. Unaudited ranking placed the three bounded source sets 4th, 14th, and 24th of 26. MeSH could not have replaced the audit: only four pathways have a descriptor, 26.7% of records carried no headings, and a relevant heading had sensitivity 0.56 and specificity 0.97 for substantive indexed units. The most frequent failure mechanism was annotation echo, pathway names present only as enrichment or predicted-target annotations; units with enrichment language were 36% pathway-substantive, against 70% of the rest. Conclusions Only three of 26 named-pathway retrievals were ready for screening, and counts alone did not show which. Query auditing assigns each retrieval a readiness status and action. Here the main target for redesign was annotation echo, pathway names entering EV records as enrichment annotations. Before treating retrieved records as pathway evidence, a review team should rate a sample for pathway substance and EV/context relevance.