Developing and evaluating a remote, self-administered, AI-powered occupational therapy initial assessment: A crossover randomized controlled trial

BackgroundArtificial intelligence (AI) is spreading through health care, yet systems that administer assessments directly to clients remain untested in occupational therapy (OT), where the initial assessment both gathers data and builds therapeutic rapport.ObjectiveTo compare user experience of a purpose-built, remote AI-administered OT initial assessment with a human-administered assessment.MethodsThirty adults completed an AI-administered and a human-administered OT initial assessment in randomized order one to two weeks apart (crossover design), rating usability, perceived compassion, goal setting and satisfaction after each and stating an overall preference; 28 gave reflection interviews. Formats were compared within participants (Wilcoxon signed-rank, McNemar, Holm-corrected) and interviews analyzed thematically.ResultsParticipants were randomized 15 per sequence; order affected no outcome. AI usability was high (modified System Usability Scale 81.1). Perceived compassion favored the human format (median 9.90 vs 7.53; Hodges-Lehmann difference 2.39, 95% CI 1.56-3.11; p < 0.001), as did satisfaction (all p < 0.001). Agreement that goals were clear, personalized, relevant, attainable and motivating did not differ (all p ≥ 0.25), but strong agreement was more frequent after the human format on four of five items (40.0-53.3 percentage points, all p ≤ 0.002). Overall, 83% (25/30) preferred the human format. Four themes were identified.ConclusionsThe AI-administered assessment was highly usable and produced goals accepted as clear, relevant and attainable at rates matching the human format; what differed was the depth of endorsement, perceived compassion and preference. Conversational AI appears better placed to supplement therapist-led assessment than to replace it.