A wearable hybrid haptic feedback system with sensorless force estimation for robot-assisted minimally invasive surgery

Force feedback is crucial in ensuring patient safety during surgery. This, however, is lost in Robot-assisted Minimally Invasive Surgery, depending instead on the practitioner’s ability to judge the applied forces. This work proposes a real-time, wearable pipeline that integrates sensorless force estimation based on monocular video frames with a hybrid visual-vibrotactile feedback system. The system is evaluated in an open-loop configuration as a foundational stage toward closed-loop deployment, using pre-recorded datasets. The force estimation algorithm builds upon recent literature, employing a vision-state estimator with an encoder–decoder architecture designed to achieve high inference rates while maintaining competitive accuracy. A concise parameter-occlusion analysis confirms that the estimator leverages robot-state cues in addition to vision. The haptic feedback system constitutes a successful proof-of-concept and requires further testing to enhance its implementation.