Evaluating ASD in children through automatic analysis of paintings

Autism spectrum disorder (ASD) is a hereditary neurodevelopmental disorder affecting individuals, families, and societies worldwide. Screening for ASD relies on specialized medical resources, and current machine learning-based screening methods depend on expensive professional devices and algorithms. Therefore, there is a critical need to develop accessible and easily implementable methods for ASD assessment. In this study, we are committed to finding such an ASD screening and rehabilitation assessment solution based on children’s paintings. From an ASD painting database, 375 paintings from children with ASD and 160 paintings from typically developing children were selected, and a series of image signal processing algorithms based on typical characteristics of children with ASD were designed to extract features from images. The effectiveness of extracted features was evaluated through statistical methods , and they were then classified using a support vector machine (SVM) and XGBoost (eXtreme Gradient Boosting). In 5-fold cross-validation, the SVM achieved a recall of 94.93%, a precision of 86.40%, an accuracy of 85.98%, and an AUC of 90.90%, while the XGBoost achieved a recall of 96.27%, a precision of 93.78%, an accuracy of 92.90%, and an AUC of 98.00%. This efficacy persists at a high level even during additional validation on a set of newly collected paintings. Not only did the performance surpass that of participated human experts, but the high recall rate, as well as its affordability, manageability, and ease of implementation, indicates potentiality in wide screening and rehabilitation assessment. All analysis code is public at GitHub: dishangti/ASD-Painting-Pub . • Design algorithms for extracting ASD painting features related to information, color, and structure. • Analyze feature correlations with autism symptoms through statistical methods. • Employ machine learning models to classify paintings and achieve a performance higher than human experts. • Analyze extracted feature importance in the models using SHAP values.