Can large language models provide reliable information on harmful skincare practices? A comparison of ChatGPT-5.2, Gemini 3, and Perplexity Sonar in debunking viral dermatological myths
Viral skincare misinformation on social media promotes practices with cutaneous toxicity risks, including irritant contact dermatitis, photosensitization, dyspigmentation, and photocarcinogenic exposure. Free-tier large language models (LLMs) increasingly answer lay skincare queries, yet their debunking performance remains largely unexamined. To compare accuracy, safety-referral behavior, and bibliographic integrity of three widely adopted free-tier LLMs (ChatGPT-5.2, Gemini 3, Perplexity Sonar) responding to patient-voice viral dermatological myths. In this cross-sectional study, twenty myths meeting predefined viral, exposure, dermatotoxicological, and testability criteria were selected by expert consensus from Google Trends and short-form video platforms (July–December 2025). Standardized conversational English prompts were submitted under default free-tier settings. Two blinded board-certified dermatologists independently scored the 60 responses for accuracy (1–5 Likert), safety referral (binary), and bibliographic integrity (verifiable vs hallucinated; peer-review status), resolving disagreements by consensus. Inter-rater agreement was substantial (κ = 0.79). Accuracy did not differ significantly (all medians 4.0; Kruskal–Wallis p = 0.44), whereas safety-referral rates did (ChatGPT 75%, Gemini 90%, Perplexity 55%; p = 0.043; Cramér’s V = 0.32). Gemini hallucinated 62.5% of references; Perplexity reached 95.0% retrieval fidelity, yet only 19.7% of its verifiable citations were peer-reviewed, versus 76.9% (ChatGPT) and 58.3% (Gemini; p < 0.001). All models produced broadly accurate refutations but failed in non-overlapping, dermatotoxicologically meaningful ways. Gemini’s frequent hallucination and Perplexity’s high-fidelity yet largely non-peer-reviewed retrieval illustrate a hallucination paradox: retrieval accuracy is necessary but insufficient for clinical or toxicological reliability. These exploratory findings suggest that patient-facing LLMs warrant cutaneous-safety–aware governance.
Authors
- Orhan Şen
Publication Details
- Journal
- Figshare
- Published
- 2026-11-11
- DOI
- https://doi.org/10.6084/m9.figshare.34983635
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00