Introducing Large Language Models in Communication and Public Relations Education: A Mixed-Methods Pilot Study

Abstract This mixed-methods pilot study investigates the potential of large language models (LLMs), specifically ChatGPT 3.5, to assist communication and public relations students in improving academic writing, focusing on clarity, conciseness, and coherence—skills aligned with cognitive load theory. An analysis of 60 abstracts (30 student-drafted abstracts and 30 AI-augmented abstracts) through qualitative and quantitative methods revealed modest enhancements. Two blinded expert evaluators identified statistically significant improvements in clarity and conciseness (Evaluator 1: p = 0.0024; Evaluator 2: p = 0.0462), although evaluator variability highlighted assessment subjectivity. Without a control group, causal claims are limited, and training sessions may have enhanced students’ proficiency with ChatGPT, complicating attribution. The students perceived greater benefits (mean improvement = 1.2) than the evaluators confirmed (mean = 0.52), suggesting potential overconfidence. Risks, including overreliance on AI and threats to originality, underscore the need for ethical guidelines, such as critical evaluation training. Owing to its small, homogeneous sample size and short duration, this study lacks demographic diversity and longitudinal insight, thus necessitating broader, long-term research.