Octargon V11.0: A Multimodal AI-Powered Gesture and Voice Controlled Desktop Interaction System

Octargon V11.0 is a multimodal desktop interaction system designed to enable hands-free control of presentations, documents, and applications through gesture recognition, voice interaction, and distributed document transfer. The system integrates perception modules, artificial intelligence processing, and network communication into a unified desktop environment that allows users to interact with digital content in an intuitive and natural way. The system operates by combining visual sensing, audio processing, and intelligent command execution. Hand gestures are detected using computer vision techniques based on MediaPipe hand landmark tracking, supported by depth filtering from the Intel RealSense D455 camera. This enables reliable detection of gestures such as swipe navigation, pinch-based zoom control, mouse movement, and gesture-triggered document transfer commands. Voice interaction is provided through an AI assistant named Bee AI, which processes spoken instructions using a speech pipeline that includes audio capture, voice activity detection, Whisper-based speech transcription, and AI command interpretation. This allows users to control system functions through natural language commands such as opening applications, navigating documents, adjusting system volume, and initiating document transfer operations. The system also incorporates a distributed document transfer mechanism that enables wireless sharing of files between computers on the same network. A Raspberry Pi device functions as an intermediate relay server that coordinates communication between sender and receiver systems, ensuring reliable document exchange and simplified device discovery. All interaction modules are integrated into a desktop interface developed using PyQt5, which provides real-time feedback on gesture detection, voice interaction status, transfer progress, and system activity logs. By combining gesture interaction, voice-based AI control, and distributed communication capabilities, Octargon V11.0 provides a unified intelligent interaction platform suitable for presentation environments, educational settings, and digital exhibition systems.