Hierarchical Learning Based on Visual-Haptic Perception for Robotic Variable Impedance Control

Dexterous manipulation in unstructured environments poses significant challenges due to the limitations of traditional robotic control methods, which often rely on precise kinematic and dynamic models with predefined behaviors, impeding adaptability and flexibility. This article presents a hierarchical control framework based on multimodal perception that integrates visual and force feedback for robotic variable impedance control. Unlike conventional approaches, the proposed framework incorporates a novel visual-feature-driven reward mechanism and dynamically learned deep kinematic and dynamic models. The high-level planner integrates visual and haptic information to optimize task execution, while the low-level controller employs a variable impedance strategy to adapt stiffness and compliance in response to task demands. Experimental validation is conducted on both simulated and real-world platforms, involving complex assembly tasks such as peg-in-hole insertion and plug-socket connections. Results demonstrate that the proposed approach significantly improves task success rates, adaptability, and robustness compared to existing methods. This article provides a comprehensive evaluation of visual-haptic integration for enhancing efficiency and generalization in contact-rich robotic manipulation tasks.