A hybrid approach combining images and questionnaires for early detection and severity assessment of Autism Spectrum Disorder

In this research, we propose a novel integrated system for the early diagnosis and cognitive enhancement of infants with Autism Spectrum Disorder (ASD). The system combines two core modules: the Behavioral Analytic Module and the Cognitive Skill Enhancement Module. The Behavioral Analytic Module includes a Questionnaire Analysis Sub-module, which utilizes Random Forest classifiers to analyze questionnaire data, and an Image Analysis Sub-module, which employs a fine-tuned VGG16 Convolutional Neural Network to process facial images. These sub-modules independently assess ASD likelihood and combine their outputs to generate a comprehensive diagnosis using a weighted averaging technique. The Cognitive Skill Enhancement Module integrates interactive games and web-based animations designed to improve cognitive abilities and daily living skills in toddlers with ASD. Additionally, a reward system is incorporated to reinforcement learning outcomes, adaptively calculating rewards based on the infants’ progress. The proposed system aims to provide a holistic approach to ASD diagnosis and intervention, offering an effective tool for early detection and tailored cognitive development. The system’s efficacy is demonstrated through comparative analysis, showing a 93% improvement in diagnostic accuracy and a 92% enhancement in cognitive skill development among toddlers with ASD. • An integrated system for early autism diagnosis combines images and questionnaires. • A new system for detecting ASD. • A new algorithm to predict the severity of ASD using questionnaire responses. • Interactive games and visual aids enhance social and cognitive skills in toddlers.