The Use of Artificial Intelligence in the Clinical Supervision of Counselors in Training: A Comparative Quantitative Study

To address the need to improve the effectiveness of clinical supervision, this study compared the use of artificial intelligence (AI) to evaluate the counseling skills of counselors in training with evaluations completed by counseling professors. Counseling session transcripts obtained from the public domain were evaluated using AI by asking the chatbot to act as if it were a licensed mental health clinical supervisor and evaluate the transcript using the Counseling Competencies Scale Revised rubric. Counseling professors were asked to manually evaluate the sessions using the same rubric. Finally, doctoral students enrolled in a counselor education and supervision program were asked to do a blind review by comparing the AI-generated evaluations with those completed by counseling professors. Students were asked to use a standardized rating system to determine which method provided more helpful feedback from a student’s perspective. By evaluating counseling transcripts and supervisor feedback in this way, the use of AI was examined as a tool for supporting the clinical supervision of counselors in training. Findings indicated AI consistently rated counseling transcripts higher than counseling professors did, indicating the two groups approached the tasks differently. Doctoral students found AI feedback more helpful in one of five evaluation competencies: balance of feedback. However, they found professor feedback more helpful in the other four categories: clarity of feedback, practicality of feedback, relevance to the counseling profession, and depth of feedback. Study results indicate further research is needed before implementing AI evaluation of counseling transcripts on a large scale. This baseline study yields valuable knowledge for the counselor education field regarding the use of AI in the clinical supervision of counselors in training. Key ideas for further research and development emerged.