Software-Defined Context-Aware Virtual Fixture for Teleoperation via Vision-Language Model

Abstract Precise teleoperation in unstructured environments remains challenging, because human errors and system-inherent limitations such as control jittering can significantly degrade task success and operational efficiency. To address these limitations, this paper proposes a software-defined Context-Aware Virtual Fixture (CAVF) system that leverages the open-world understanding of a Vision-Language Model (VLM). The proposed system automatically generates motion guidance for the robot end-effector (EEF) to assist teleoperation, preserving critical task-relevant motion while suppressing unnecessary translation and rotation components. This framework enables the autonomous generation of context-aware, haptic-free assistance without relying on handcrafted rules or physical force-feedback devices. Quantitative experiments in both simulation and real-world environments demonstrate that the proposed system significantly improves success rates, reduces completion times, and enhances trajectory efficiency compared to conventional teleoperation.