DETECTING SYNTHETIC EVIDENCE CONTAMINATION IN AI-ASSISTED SCIENTIFIC LITERATURE THROUGH PROVENANCE-AWARE LEARNING

The increasing adoption of artificial intelligence (AI) and large language models (LLMs) in scientific research has transformed literature synthesis, citation generation, evidence interpretation, and scholarly communication. However, the growing dependence on AI-assisted research tools has introduced significant challenges involving fabricated references, unsupported numerical findings, citation inconsistencies, and unverifiable scientific claims. These problems contribute to synthetic evidence contamination, potentially undermining scientific credibility, research reproducibility, and evidence-based decision-making. This study proposes an integrated Provenance-Aware Learning Framework (PALF) for identifying and evaluating synthetic evidence contamination through machine learning, natural language processing, scientific claim verification, and bibliographic provenance analysis. The proposed framework integrates conventional machine learning classifiers, transformer-based semantic representations, citation metadata verification, and source traceability indicators to distinguish verifiable scientific evidence from potentially fabricated or unsupported research content. The study adopts a quantitative comparative research design using publicly available scientific evidence verification resources, including SciFact, PubMed, Crossref, and OpenAlex. The experimental framework compares Logistic Regression, Support Vector Machine, Random Forest, Gradient Boosting, transformer-based classification, and provenance-aware learning using accuracy, precision, recall, F1-score, ROC-AUC, and false-positive rate. The original SciFact benchmark contains 1,409 scientific claims and 5,183 scientific abstracts, providing a documented numerical foundation for scientific claim verification. Additional experimental procedures are designed to evaluate fabricated citations, unsupported quantitative statements, citation-claim inconsistencies, and source metadata discrepancies. The proposed numerical evaluation includes 10 comparative tables and three research figures covering dataset characteristics, contamination distributions, classification performance, citation verification, provenance-feature contributions, and detection robustness. The study investigates whether combining semantic representations with bibliographic provenance improves evidence verification compared with text-only classification approaches. By incorporating interpretable source-validation indicators, the framework aims to support transparent identification of suspicious scientific claims and reduce dependence on linguistic detection alone. Although benchmark dataset statistics are documented, comparative model outcomes will require experimental validation. The proposed research contributes an integrated and reproducible approach to scientific evidence integrity assessment, emphasizing verifiable citations, numerical consistency, source authenticity, and explainable detection decisions. Its potential applications include automated manuscript screening, systematic literature review validation, academic publishing quality assurance, and responsible AI-assisted scientific research.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24273515
Primary Topic
Misinformation and Its Impacts
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

DETECTING SYNTHETIC EVIDENCE CONTAMINATION IN AI-ASSISTED SCIENTIFIC LITERATURE THROUGH PROVENANCE-AWARE LEARNING

Divya Vishwanath
Zenodo (CERN European Organization for Nuclear Research)
Misinformation and Its Impacts
article

DETECTING SYNTHETIC EVIDENCE CONTAMINATION IN AI-ASSISTED SCIENTIFIC LITERATURE THROUGH PROVENANCE-AWARE LEARNING

Divya Vishwanath
article en

Abstract

The increasing adoption of artificial intelligence (AI) and large language models (LLMs) in scientific research has transformed literature synthesis, citation generation, evidence interpretation, and scholarly communication. However, the growing dependence on AI-assisted research tools has introduced significant challenges involving fabricated references, unsupported numerical findings, citation inconsistencies, and unverifiable scientific claims. These problems contribute to synthetic evidence contamination, potentially undermining scientific credibility, research reproducibility, and evidence-based decision-making. This study proposes an integrated Provenance-Aware Learning Framework (PALF) for identifying and evaluating synthetic evidence contamination through machine learning, natural language processing, scientific claim verification, and bibliographic provenance analysis. The proposed framework integrates conventional machine learning classifiers, transformer-based semantic representations, citation metadata verification, and source traceability indicators to distinguish verifiable scientific evidence from potentially fabricated or unsupported research content. The study adopts a quantitative comparative research design using publicly available scientific evidence verification resources, including SciFact, PubMed, Crossref, and OpenAlex. The experimental framework compares Logistic Regression, Support Vector Machine, Random Forest, Gradient Boosting, transformer-based classification, and provenance-aware learning using accuracy, precision, recall, F1-score, ROC-AUC, and false-positive rate. The original SciFact benchmark contains 1,409 scientific claims and 5,183 scientific abstracts, providing a documented numerical foundation for scientific claim verification. Additional experimental procedures are designed to evaluate fabricated citations, unsupported quantitative statements, citation-claim inconsistencies, and source metadata discrepancies. The proposed numerical evaluation includes 10 comparative tables and three research figures covering dataset characteristics, contamination distributions, classification performance, citation verification, provenance-feature contributions, and detection robustness. The study investigates whether combining semantic representations with bibliographic provenance improves evidence verification compared with text-only classification approaches. By incorporating interpretable source-validation indicators, the framework aims to support transparent identification of suspicious scientific claims and reduce dependence on linguistic detection alone. Although benchmark dataset statistics are documented, comparative model outcomes will require experimental validation. The proposed research contributes an integrated and reproducible approach to scientific evidence integrity assessment, emphasizing verifiable citations, numerical consistency, source authenticity, and explainable detection decisions. Its potential applications include automated manuscript screening, systematic literature review validation, academic publishing quality assurance, and responsible AI-assisted scientific research.

Zenodo (CERN European Organization for Nuclear Research)
Misinformation and Its Impacts
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon