Autism Detection Using Self-Stimulatory Behaviors

This paper introduces a novel video-audio-based model for the early detection of Autism Spectrum Disorder (ASD), focusing on analyzing self-stimulatory behaviors (stim- ming) such as arm flapping, head banging, and spinning, which are critical diagnostic markers. Traditional diagnostic approaches often depend on subjective clinical observations, leading to in- consistencies, delays, and limited accessibility in diverse settings. The proposed model combines video analysis with audio detection to address these shortcomings, supported by a generative AI- based method to create an audio dataset. This dataset enhances the model’s robustness by incorporating diverse audio features merged with video data to provide a comprehensive analysis. Video analysis employs YOLO for face detection, MediaPipe for facial landmarks, pose tracking, and gaze estimation, while CNN- LSTM models identify repetitive behaviors. Audio processing extracts features via Librosa, NoiseReduce, and PyDub, with Gradient Boosting models analyzing speech anomalies. Clas- sification integrates CNN-LSTM for video, Gradient Boosting for audio, and a Stacking Classifier with Logistic Regression for final predictions. By merging video and audio cues, this model enhances the objectivity, accuracy, and scalability of ASD detection, providing a reliable and efficient framework for early diagnosis and intervention across varied real-world environments.