Work + Micro-Exercise: A Multi-Modal Human-Computer Interaction System Using MediaPipe for Mouse Replacement and Ergonomic Well-Being

Sedentary work culture in corporate environments often results in musculoskeletal discomfort, eye strain, and repetitive stress injuries. To address these challenges, we present a <i>novel Human-Computer Interaction (HCI) system</i> that replaces the conventional mouse with <i>eye blinks, head movements, and hand gestures</i> using <i>MediaPipe and OpenCV</i>. Our system enables employees to perform <i>micro-exercises</i> during regular computer use without interrupting productivity—eye blinks are mapped to clicks, head nods to scrolling, and hand gestures to cursor control, drawing, and erasing. The proposed solution was benchmarked against five widely used pose detection frameworks—<i>MediaPipe, OpenPose (CMU), MoveNet (TensorFlow.js), DeepLabCut, and AlphaPose</i>—on parameters including <i>tracking accuracy, processing speed, ease of deployment, and computational load</i>. Results show that MediaPipe achieved the most balanced performance with <i>92% tracking accuracy, ~25 fps real-time responsiveness, and minimal system overhead on standard webcams</i>, making it ideal for office deployment. OpenPose provided higher accuracy (~95%) but required GPU acceleration, resulting in <i>significant processing delays (~8-10 fps)</i>. MoveNet delivered <i>lightweight, browser-compatible performance (20-22 fps)</i> but with limited keypoint coverage. DeepLabCut offered <i>high customizability and ~93% accuracy</i> but required dataset-specific training, making it less suitable for generic office applications. AlphaPose performed competitively (~94% accuracy, ~18 fps) but required higher-end hardware for stable operation. To validate ergonomic benefits, a <i>post-usage ergonomic comfort chart (System Usability Scale + self-reported musculoskeletal feedback)</i> was collected from <i>10 IT professionals</i> over a two-week trial. Findings indicated <i>40% reduction in reported neck stiffness, 35% lower hand fatigue, and 25% improved eye engagement</i> compared to baseline mouse usage. Users rated the system at <i>78/100 on usability</i>, supporting its dual role in <i>productivity and workplace health</i>. This work demonstrates that <i>webcam-based multimodal mouse replacement</i> can be a <i>low-cost, scalable, and health-oriented alternative</i> to conventional input devices. Beyond corporate ergonomics, the approach has strong potential for <i>accessibility in assistive technologies</i> and as a <i>CSR initiative</i> for inclusive digital workplaces.