SiBot: A hybrid framework for fine-grained semantic attribution of AI-generated text and its production deployment

Large language models such as ChatGPT, Gemini, Claude and DeepSeek now generate text that is routinely indistinguishable from human writing, yet virtually every deployed detector answers only one binary question: human or AI. This is the wrong question for the settings that matter most. Digital forensics needs to know which model produced a text; academic-integrity adjudication needs to know whether a human was involved at all; and neither can be answered by a single probability, least of all for the hybrid human-AI drafts that now dominate real-world writing. This paper addresses that gap with SiBot, an end-to-end framework that reformulates AI-text detection as fine-grained multi-class source attribution and carries it through to public deployment. We construct two purpose-built corpora using a controlled bank of 3,000 prompts spanning six subject domains and five task types: a 66,000-sample corpus covering 22 AI sources, and a 30,000-sample corpus covering 10 classes that explicitly models five pure sources and four human-AI collaboration workflows; both are publicly released as the TXD-22 benchmark dataset. On the 22-class corpus we benchmark twelve feature-classifier combinations and find that TF-IDF with Random Forest attains 68.2% accuracy and an AUC of 0.742. On the 10-class corpus we benchmark fifteen classical, deep-learning and fine-tuned transformer baselines, and propose a hybrid architecture that fuses sparse TF-IDF lexical features with DeBERTa contextual embeddings at the feature level before Random Forest classification. The proposed hybrid attains 92.38% accuracy, 92.41% precision, 92.38% recall and 92.31% F1 under prompt-aware 10-fold cross-validation, a 17.73-point gain over the strongest fine-tuned transformer (DeBERTa, 74.65%) and a 34.30-point gain over TF-IDF with Random Forest alone, with non-overlapping 95% confidence intervals. An ablation confirms that both branches are necessary, and SHAP and LIME analyses expose the lexical evidence behind individual decisions. Because the 10-class hybrid is both substantially more accurate and interpretable, it is the model served by the live SiBot platform; the 22-class model is reported here as a research benchmark, and extending the deployment to all 22 classes is left to future work. In a controlled three-way benchmark against QuillBot, Turnitin, ZeroGPT and TextGuard, SiBot is the only system that names the generating model of AI text (Gemini, 96.7% confidence) and the only one that recognises hybrid provenance (97.7% confidence, against verdicts ranging from 0% to 76% among the commercial tools on the identical input).

Authors

Institutions

Publication Details

Journal
PLoS ONE
Published
2026-09-08
DOI
https://doi.org/10.1371/journal.pone.0357405
Primary Topic
Misinformation and Its Impacts
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

SiBot: A hybrid framework for fine-grained semantic attribution of AI-generated text and its production deployment

Md. Sadiq Iqbal, Mohammod Abul Kashem
PLoS ONE
Misinformation and Its Impacts
article

SiBot: A hybrid framework for fine-grained semantic attribution of AI-generated text and its production deployment

Md. Sadiq Iqbal, Mohammod Abul Kashem
article en

Abstract

Large language models such as ChatGPT, Gemini, Claude and DeepSeek now generate text that is routinely indistinguishable from human writing, yet virtually every deployed detector answers only one binary question: human or AI. This is the wrong question for the settings that matter most. Digital forensics needs to know which model produced a text; academic-integrity adjudication needs to know whether a human was involved at all; and neither can be answered by a single probability, least of all for the hybrid human-AI drafts that now dominate real-world writing. This paper addresses that gap with SiBot, an end-to-end framework that reformulates AI-text detection as fine-grained multi-class source attribution and carries it through to public deployment. We construct two purpose-built corpora using a controlled bank of 3,000 prompts spanning six subject domains and five task types: a 66,000-sample corpus covering 22 AI sources, and a 30,000-sample corpus covering 10 classes that explicitly models five pure sources and four human-AI collaboration workflows; both are publicly released as the TXD-22 benchmark dataset. On the 22-class corpus we benchmark twelve feature-classifier combinations and find that TF-IDF with Random Forest attains 68.2% accuracy and an AUC of 0.742. On the 10-class corpus we benchmark fifteen classical, deep-learning and fine-tuned transformer baselines, and propose a hybrid architecture that fuses sparse TF-IDF lexical features with DeBERTa contextual embeddings at the feature level before Random Forest classification. The proposed hybrid attains 92.38% accuracy, 92.41% precision, 92.38% recall and 92.31% F1 under prompt-aware 10-fold cross-validation, a 17.73-point gain over the strongest fine-tuned transformer (DeBERTa, 74.65%) and a 34.30-point gain over TF-IDF with Random Forest alone, with non-overlapping 95% confidence intervals. An ablation confirms that both branches are necessary, and SHAP and LIME analyses expose the lexical evidence behind individual decisions. Because the 10-class hybrid is both substantially more accurate and interpretable, it is the model served by the live SiBot platform; the 22-class model is reported here as a research benchmark, and extending the deployment to all 22 classes is left to future work. In a controlled three-way benchmark against QuillBot, Turnitin, ZeroGPT and TextGuard, SiBot is the only system that names the generating model of AI text (Gemini, 96.7% confidence) and the only one that recognises hybrid provenance (97.7% confidence, against verdicts ranging from 0% to 76% among the commercial tools on the identical input).

PLoS ONEVol. 21(9)
Dhaka University of Engineering & Technology (BD)
University of Engineering and Technology, Lahore, University of Dhaka
Quality Education
Openalex Percentile: Top 5%
Misinformation and Its Impacts
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.