Expert-Rated Quality of AI-Assisted Clinical Supervision in Strategic Psychotherapy A Pilot Study
This poster presents a single-centre exploratory pilot study assessing the clinical quality and model-fidelity of AI-assisted clinical supervision feedback in strategic-systemic psychotherapy. Feedback was generated by a purpose-built configuration of Claude Opus 5 (Anthropic) — a systemic-strategic system prompt combined with a curated knowledge base of systemic-therapy texts, including Argentine authors — evaluated against a general-purpose model baseline. Fifteen real, anonymized supervision cases were rated by 15 blinded expert supervisors (3 raters per case; 45 evaluations total) across five anchored clinical dimensions: theoretical relevance, clinical applicability, safety/risk, model adherence, and usefulness. All feedback shown to raters was AI-generated; raters were told any item might be human-written in AI style. Blinded experts rated the AI's supervisory feedback as clinically appropriate in 89% of evaluations (95% CI 77–95) and recommended 91% of outputs for delivery to the therapist. Raters could not reliably identify the feedback as AI-generated (63% classified as AI, p = 0.13), and a secondary finding showed significantly higher quality ratings when feedback was attributed to a human (Mann–Whitney p < 0.001; rank-biserial r = 0.58) — consistent with a "good = human" attribution bias. Presented at the World Congress of Psychiatry (WCP) 2026. Limitations: single-centre pilot; one AI configuration; ceiling effect; rater severity; clustering not yet modelled; rubric not validated. A controlled non-inferiority study is planned as the next step.
Authors
- Sergio Alejandro Grosman (ORCID: https://orcid.org/0009-0004-6521-8307)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-05
- DOI
- https://doi.org/10.5281/zenodo.22416009
- Primary Topic
- Digital Mental Health Interventions
- Type
- article
- Field-Weighted Citation Impact
- 0.00