Interaction Detection in Images of Therapy Sessions with Children with Autism Spectrum Disorder

Autism spectrum disorder (ASD) affects many children and limits their social interaction, communication, and behavioral skills. Regular follow-ups with qualified professionals assist in the patients' development through sessions and progress evaluations. This progress is manually recorded by professionals, which can lead to errors in analysis. To assist with these annotations, an automation process is proposed using pose estimation and object detection techniques in computer vision to detect interaction between participants in therapy sessions with children with ASD. For this purpose, the Yolov8 object detector and Yolov8-Pose estimator are applied. Subsequently, interaction detection is performed using the results obtained from the previous predictions. Heuristics were introduced to compare these techniques. To enhance performance, solutions were explored to address the limitations encountered in object detection and pose estimation predictions. The results demonstrated a 15.8% improvement in precision for the pose estimation heuristic compared to the bounding box heuristic, achieving satisfactory performance for the proposed approach.