Unified framework for in-hand telemanipulation and haptic feedback in real-world deployment

Dexterous robotic hands offer the ability to perform fine-grained manipulation within human-centered environments, yet achieving robust in-hand telemanipulation in real-world settings remains challenging due to uncertain contact dynamics, limited operator feedback, and difficulty maintaining performance across long-horizon tasks. This paper presents a unified framework for real-world in-hand telemanipulation with integrated haptic feedback that combines human teleoperation and autonomous dexterous manipulation. Human manipulation intent is represented using the End-Effects-Oriented Learning-Based Dexterous Telemanipulation (EFOLD) framework, which extracts object-centric end-effector interaction modalities to supervise a deep reinforcement learning policy for in-hand manipulation. To enhance operator situational awareness, we introduce a custom momentum-based haptic device provides real-time feedback of object rotational state and interaction forces. To ensure reliable and continuous task execution across long-horizon manipulation, we develop an adaptive policy switching strategy that dynamically reallocates control authority between operator teleoperation and autonomous manipulation. This mechanism enables seamless transitions across manipulation stages and maintains stability under varying contact conditions. Real-world evaluations demonstrate that the proposed framework supports dynamic, high-precision in-hand telemanipulations and improved operator's performance with robustness.