Interpreting Self-Reported Outcomes After AI-Assisted Writing Workshops: An Aggregate Sensitivity Analysis of Educational Claims

Artificial intelligence writing workshops are increasingly used in health-professions education, but immediate self-reports can be interpreted as evidence of learning or ethical competence beyond what they measure. This study applied an aggregate-data interpretation framework to a published single-institution evaluation of a hands-on AI-assisted academic-writing workshop. Public counts and item summaries were classified by construct, summarized with binomial working-model intervals for observed respondents, and reanalyzed with complete registered-list identification intervals that assigned every questionnaire nonrespondent first to the non-positive and then to the positive category. Among 64 respondents, 54 reported improved understanding, or 84.4 percent, with a working-model interval of 73.6 percent to 91.3 percent. Confidence and perceived ethical improvement were each reported by 45 respondents, or 70.3 percent, with intervals of 58.2 percent to 80.1 percent. Across 80 registrants, unrestricted nonresponse yielded identification intervals of 67.5 percent to 87.5 percent for perceived improvement and 56.2 percent to 76.2 percent for both confidence and ethical improvement. Positive delivery ratings were common. However, these measures did not assess objective knowledge, writing quality, source verification, ethical decision-making, or sustained practice. The evidence supports bounded descriptions of respondent experience only; effectiveness claims require linked objective assessment and delayed follow-up.