Authors’ Reply: Citation Accuracy Challenges Posed by Large Language Models

Large language models (LLMs) have demonstrated significant potential in academic research but face challenges in generating accurate citations. The issue of hallucinated references—well-formatted but fictitious citations—arises due to LLMs' limited access to subscription-based databases and their reliance on probabilistic text generation. This letter discusses two key approaches to mitigating these issues. First, retrieval-augmented generation (RAG) combined with Hallucination Aware Tuning (HAT) improves citation integrity by integrating external databases and employing hallucination detection models. However, even RAG-HAT systems may still misinterpret source content. Second, we propose the development of “Reference-Accurate” Academic LLMs by major global publishers, which would be trained exclusively on rigorously verified academic literature, ensuring that all citations generated are authentic and traceable. We recommend a dual approach integrating RAG-HAT with publisher-backed academic LLMs, along with human oversight, to enhance AI-assisted scholarly communication. Future research should evaluate the accuracy and reliability of these methods to promote responsible AI use in academia.

Authors

Institutions

Publication Details

Journal
JMIR Medical Education
Published
2025-04-02
DOI
https://doi.org/10.2196/73698
Citations
2
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
1.31
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Authors’ Reply: Citation Accuracy Challenges Posed by Large Language Models

Khalid H. Malki, Ayman Al‐Eyadhy, Khalid Alhasan, Amr Jamal et al.
2 citations
JMIR Medical Education
Artificial Intelligence in Healthcare and Education
1.31
article

Authors’ Reply: Citation Accuracy Challenges Posed by Large Language Models

Khalid H. Malki, Ayman Al‐Eyadhy, Khalid Alhasan, Amr Jamal, Mohamad‐Hani Temsah
article en
2 citations

Abstract

Large language models (LLMs) have demonstrated significant potential in academic research but face challenges in generating accurate citations. The issue of hallucinated references—well-formatted but fictitious citations—arises due to LLMs' limited access to subscription-based databases and their reliance on probabilistic text generation. This letter discusses two key approaches to mitigating these issues. First, retrieval-augmented generation (RAG) combined with Hallucination Aware Tuning (HAT) improves citation integrity by integrating external databases and employing hallucination detection models. However, even RAG-HAT systems may still misinterpret source content. Second, we propose the development of “Reference-Accurate” Academic LLMs by major global publishers, which would be trained exclusively on rigorously verified academic literature, ensuring that all citations generated are authentic and traceable. We recommend a dual approach integrating RAG-HAT with publisher-backed academic LLMs, along with human oversight, to enhance AI-assisted scholarly communication. Future research should evaluate the accuracy and reliability of these methods to promote responsible AI use in academia.

JMIR Medical EducationVol. 11
King Saud Medical City (SA), King Saud University (SA)
Openalex Percentile: Top 17%
Artificial Intelligence in Healthcare and Education
1.31
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.