Assessing AI-generated smoking cessation advice for patient education in primary care

BACKGROUND: Artificial intelligence (AI) has emerged as a promising tool to support smoking cessation in primary care, particularly for populations underserved by traditional interventions. However, the quality of AI-generated smoking cessation advice remains understudied, especially in low-resource settings and among vulnerable groups such as adolescents. OBJECTIVE: This study aims to evaluate AI-generated responses to smoking cessation questions for patient education in primary care, comparing different AI programs in terms of knowledge, readability, and quality. METHODS: Ten publicly accessible AI programs were prompted in Turkish with 24 standardized, open-ended smoking cessation questions framed as a patient consultation. Two family medicine specialists independently assessed each response's readability using Ateşman's Readability Index, reliability using the DISCERN instrument, and accuracy and motivational interviewing quality using a bespoke rubric and OARS (Open questions, Affirmations, Reflections, Summaries) framework. Inter-rater agreement was evaluated via intraclass correlation. Descriptive statistics were computed for readability scores, DISCERN ratings, and accuracy grades. RESULTS: All AI programs provided at least partially correct answers to all questions. The average readability score was 54.90 (medium difficulty) according to Atesman's Index. The mean DISCERN score was 66 ± 5.2, indicating excellent quality. Three AI programs incorporated core motivational interviewing skills. The most accurately answered question concerned e-cigarettes' harm compared to traditional cigarettes, while medication advice was least evidence-based. CONCLUSIONS: Free AI chatbots deliver reliably accurate and moderately readable smoking cessation advice, supporting their potential role as patient education adjuncts in primary care-particularly for individuals with at least a high school education. Further research should compare AI-assisted versus clinician-led interventions on smoking cessation outcomes.