Evaluating AI-Generated informed consent documents in oral surgery: A comparative study of ChatGPT-4, Bard gemini advanced, and human-written consents
This study evaluates the quality and readability of informed consent documents generated by AI platforms ChatGPT-4 and Bard Gemini Advanced compared to those written by a first-year oral surgery resident for common oral surgery procedures. The evaluation, conducted by 18 experienced oral and maxillofacial surgeons, assessed consents for accuracy, completeness, readability, and overall quality. ChatGPT-4 consistently outperformed both Bard and human-written consents. ChatGPT-4 consents had a median accuracy score of 4 [IQR 4-4], compared to Bard's 3 [IQR 3-4] and human's 4 [IQR 3-4]. Completeness scores were higher for ChatGPT-4 (4 [IQR 4-5]) than Bard (3 [IQR 3-4]) and human (4 [IQR 3-4]). Readability was also superior for ChatGPT-4, with a median score of 4 [IQR 4-5] compared to Bard and human consents, both at 4 [IQR 4-4] and 4 [IQR 3-4], respectively. The Gunning Fog Index for ChatGPT-4 was 17.2 [IQR 16.5-18.2], better than Bard's 23.1 [IQR 20.5-24.7] and the human consents' 20 [IQR 19.2-20.9]. Overall, ChatGPT-4's consents received the highest quality ratings, underscoring AI's potential in enhancing patient communication and the informed consent process. The study suggests AI can reduce misinformation risks and improve patient understanding, but continuous evaluation, oversight, and patient feedback integration are crucial to ensure the effectiveness and appropriateness of AI-generated content in clinical practice.
Authors
- Guido Gabriele (ORCID: https://orcid.org/0000-0002-1028-7493)
- Fabiana Allevi (ORCID: https://orcid.org/0000-0002-5012-4031)
- Vincenzo Abbate (ORCID: https://orcid.org/0000-0002-7905-0531)
- Giovanni Salzano (ORCID: https://orcid.org/0000-0003-3947-8733)
- Alberto Maria Saibene (ORCID: https://orcid.org/0000-0003-1457-6871)
- Andrea Frosolini (ORCID: https://orcid.org/0000-0003-1347-4013)
- Resi Pucci (ORCID: https://orcid.org/0000-0001-6237-2334)
- Fabio Maglitto (ORCID: https://orcid.org/0000-0002-1080-4067)
- Stefania Troise (ORCID: https://orcid.org/0000-0001-8421-0328)
- Marzia Petrocelli (ORCID: https://orcid.org/0000-0003-1798-3595)
- Luigi Angelo Vaira (ORCID: https://orcid.org/0000-0002-7789-145X)
- Antonino Maniaci (ORCID: https://orcid.org/0000-0002-1251-0185)
- Antonio Arena (ORCID: https://orcid.org/0000-0003-0903-3733)
- Salvatore Crimi (ORCID: https://orcid.org/0000-0002-2374-9853)
- Giacomo De Riu (ORCID: https://orcid.org/0000-0002-7201-6445)
- Valentino Vellone (ORCID: https://orcid.org/0000-0001-6048-1190)
- Michela Bergonzani (ORCID: https://orcid.org/0000-0002-1646-0509)
- Giada Anna Beltramini (ORCID: https://orcid.org/0000-0002-4351-0276)
- Luigi Orrù (ORCID: https://orcid.org/0000-0003-3496-218X)
- Jerome Rene Lechien (ORCID: https://orcid.org/0000-0002-0845-0845)
- Paolo Boscolo‐Rizzo (ORCID: https://orcid.org/0000-0002-4635-7959)
- Alessandro Tel (ORCID: https://orcid.org/0000-0001-8395-5272)
- Carlos M. Chiesa‐Estomba (ORCID: https://orcid.org/0000-0001-9454-9464)
- Alessandro Bolzoni (ORCID: https://orcid.org/0000-0003-1713-5481)
- Miguel Mayo‐Yáñez (ORCID: https://orcid.org/0000-0002-1829-6597)
- Giuseppe Tanda
Institutions
- University of Siena (IT)
- University of Parma (IT)
- University of Udine (IT)
- University of Mons (BE)
- University of Trieste (IT)
- University of Sassari (IT)
- University of Milan (IT)
- Università degli Studi di Enna Kore (IT)
- Santa Maria Nuova Hospital (IT)
- Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico (IT)
- University of Catania (IT)
- Biogipuzkoa Health Research Institute (ES)
- Ospedale San Paolo (IT)
- Complexo Hospitalario Universitario A Coruña (ES)
- Carlo Forlanini Hospital (IT)
- University of Bari Aldo Moro (IT)
- University of Naples Federico II (IT)
Publication Details
- Journal
- Journal of Cranio-Maxillofacial Surgery
- Published
- 2024-10-28
- DOI
- https://doi.org/10.1016/j.jcms.2024.10.002
- Citations
- 27
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 3.25
Funders
- Università degli Studi di Sassari