An Intelligent Mouse Control Interface Using Multimodal Interaction

In this paper, we propose a multimodal system integrating hand gesture recognition and voice commands to enhance Human-Computer Interaction (HCI) without relying on traditional input devices like a mouse. This system is particularly beneficial for accessibility applications, aiding users with limited hand control by enabling intuitive computer navigation. The system demonstrated high recognition accuracy—92 % for gestures and 95 % for voice commands—with an average response time of under 0.5 seconds, making it suitable for real-time applications. It is cost-effective and requires only a standard webcam and microphone, ensuring easy deployment without specialized hardware. The study discusses the sociocultural impact of multimodal interaction, highlighting its advantages, challenges, and future improvements for accuracy and gesture expansion. This research contributes to inclusive HCI, empowering diverse user groups with greater interaction flexibility.