ParaEyes: The Smart Eye Bot System for Paralyzed Patients

Locked-in Syndrome (LIS) poses significant challenges for individuals experiencing complete paralysis, who must rely solely on eye and head movements and hearing to communicate. The main issue arises from the urgent need for a customized communication solution due to the severe limitations LIS patients encounter in interacting with their surroundings. This difficulty extends beyond physical constraints, greatly impacting their overall quality of life and psychological well-being. To address this complex challenge, we have developed an innovative approach that integrates advanced eye-tracking technology, Natural Language Processing (NLP) and Artificial Intelligence (AI). This holistic solution not only restores communication capabilities but also provides crucial support for the mental health and psychological well-being of LIS patients, offering a ray of hope for a better future. Beyond addressing communication challenges, our proposal also focuses on improving the mental health of LIS patients through interactive communication either with surrounding people or AI bots. Our solution, named “ParaEyes” utilizes webcam-detected eye movements to navigate the communication interface, employing real-time video image processing using Python. Additionally, users can engage with various AI bots, including ChatGPT, YouTube Bot and ParaEyes Visual Bot, based on their preferences. ParaEyes achieves a notably 2X faster average setup time and on average 10% higher accuracy compared to the open-source alternatives. This approach enhances communication skills and mental health support while also being inspired by ChatGPT’s robust measures to safeguard user data and ensure user privacy through tailored interactions and responsive functionalities.