Intelligent Assisted Eating System for Parkinson's Patients Based on Multimodal Deep Learning in Complex Backgrounds

This study presents an innovative intelligent assisted feeding system aimed at improving the autonomous eating ability and quality of life of patients with Parkinson's disease. The system integrates three core technology modules: (1) a speech recognition module based on Paraformer for accurately understanding the patient's dietary needs; (2) a real-time oral detection module using YOLOv5s for precisely locating the patient's mouth and assessing its open or closed state; (3) a head pose estimation module based on FSA-Net to synchronize the robotic arm with the patient's head movement in real time. Through multimodal fusion, the system achieves full-process automation from speech command recognition to precise food delivery. Experimental results show that the system effectively identifies the patient's intent, accurately locates the mouth, and adjusts the food delivery angle in real time according to the head's posture, significantly enhancing the safety and comfort of the eating process. This research not only provides a practical feeding assistance solution for patients with Parkinson's disease but also opens new avenues for applying intelligent assistance technologies in the rehabilitation of neurodegenerative diseases.