AI-powered Personalized Online Drawing Assistant with Hand Skeleton Analysis

In this paper, we present a computer-vision-based system designed to enhance online drawing instruction by offering prompt feedback and evaluating learners' hand movements and pen handling.The system utilizes only webcam data, avoiding the need for expensive external devices.It consists of three main components: hand skeleton analysis, drawing tool isolation, and movement similarity assessment.MediaPipe is employed for hand skeleton analysis, interpreting the pen-holding postures of both the instructor and the learner.A dual-phase strategy is employed to isolate drawing tools.You Only Look Once (YOLO) v5 performs precise object detection to locate the drawing tool within the image, and U-Net is then used to separate the tool from the surrounding area in the identified region.Movement similarity is assessed using dynamic time warping and Procrustes-aligned mean per-joint position error, comparing the hand movements of the instructor and learner.This drawing assistant system is designed to effectively replicate the instructor's role in online environments, focusing on improving learning outcomes and experiences through interactivity and personalization.Experimental evaluation confirms the system's effectiveness and usability.Results demonstrate that the system provides accurate and timely feedback and assessment for online drawing instruction, thereby enhancing learners' drawing skills and learning experiences.