Neuromorphic Perception and Local Multimodal Haptic Feedback Based Immersive Teleoperation

The integration of teleoperating robots with the Internet of Things (IoT) presents opportunities in fields like remote healthcare and semi-mechanical control. However, the incompleteness of sensory feedback can lead to operator fatigue, and latency may result in danger at the execution end when manipulating sharp objects. This study introduces a closed-loop teleoperation system leveraging multisensory fusion, visual, and haptic feedback and neuromorphic perception to mitigate this issue. At the control end, real-time control is achieved through cellular modules (delay < 71ms and position tracking RMS < 7mm) and we integrate multimodal feedback to reduce operator fatigue, including visual, vibration, temperature, and electrical stimulation. Feedback from ten volunteers grasping four objects reveals a reduction in mental fatigue (42% on average) and physical fatigue (32% on average) and higher immersion. At the remote end, we use piezoresistive films and negative temperature coefficient sensors to detect force and temperature, respectively, and RGB-D cameras to capture images and depth information. Meanwhile, a neuromorphic perception system is deployed at the remote end to rapidly process unexpected events, achieving pain reflex (haptic enhanced to 500%) and slip detection (with a response time of 1 ms). These findings demonstrate the system's potential to improve user experience and operational efficiency in remote healthcare and industry.