ASDenseFormer: A Video-Based Autistic Self-Stimulatory Gesture Recognition Using DenseNet121 and Transformer Model

Complex human actions in healthcare set-tings may now be understood thanks to developments in deep learning and contactless sensors. Several computer vision-based deep learning algorithms have been developed in particular to enable a comprehensive analysis of human gesture recognition, including repet-itive movements. The most typical repeated motions that children with autism exhibit include spinning, head banging, and arm flapping. We present ASDense-Former, a hybrid model developed in this work to recognize repeated autistic movements in real time. The Self-Stimulatory Behavior Dataset (SSBD), which is available to the public, is used in this model. The Transformer model's self-attention principle handles the temporal information and classifies them, while DenseNet121 extracts spatial properties from video frames. We demonstrate that using a DensNet121 in conjunction with the Transformer model yields the best outcomes by carefully examining feature extraction and learning methodologies. Our model performed 10% better than the state-of-the-art works with an average accuracy of 86.66% in identifying the three repeating movements.