Vocal Gaze Mouse Controller

Abstract- This paper presents the design and implementation of a multimodal, real-time Human-Computer Interaction (HCI) system called the Vocal Gaze Mouse Controller. The system integrates eye-gaze tracking, blink detection, and voice recognition to control mouse functions, providing a fully hands-free interaction mechanism. Designed especially for individuals with physical disabilities, the system enables cursor navigation and command execution using webcam-based eye tracking and microphone-enabled speech processing. We use Dlib for facial landmark detection, OpenCV for image processing, PyAutoGUI for mouse control, and Python’s Speech Recognition library for interpreting voice inputs. This solution is non-invasive, affordable, and highly accurate, making it suitable for real-world accessibility applications. Results from experimental evaluations show high accuracy in blink and voice recognition, as well as low latency and user satisfaction.