Verbal C2 for Underwater In-Contact Manipulation: A Semantic Approach to Somatic Task Guidance

Effectively controlling underwater robots for in-contact manipulation remains a significant challenge. While automation excels at precision and endurance, humans offer a vocabulary and body understanding of how contact-forces are experienced, yet recent innovations in language control have not yet been applied to underwater maintenance. To address this gap, we introduce a novel Speech2Action controller that parses human voice into robot actions, searching each parsed sentence for possible matches into our 28 pre-programmed methods and parameters. To help an operator rapidly orient, we structure manipulation into three phases: “approach,” “prepare,” and “task” segments, which pre-computes environmental setup, in loop with force-sensing. The evaluation of our implementation with convenience participants demonstrated that our system can effectively perform whiteboard drawing and erasing on a compliant vertical surface, navigating through sea-inspired obstacles. While voice parsing does not guarantee correct method parsing (85% task success), we observed that participants structure commands around high and low-level goals. Future work will expand this framework to underwater evaluation and incorporate pointing and/or PADI-inspired scuba diver gesture recognition, enhancing collaboration prospective for human-in-loop underwater robots.