Accurate targeting in robot-assisted TCM pulse diagnosis using adaptive sensor fusion

Accurate targeting plays an important role in the study of human-robot interaction under dynamic environments. Especially for robot-assisted Traditional Chinese Medicine (TCM) pulse diagnosis, the localization and accuracy of diagnose positions at wrist needs to be addressed. In this work, imaging photoplethysmography (iPPG) which measures the physiological changes of blood flow in artery is used as an extra modal information in addition to computer vision at localization, to alleviate the effect of approaching distance varying during the robot arm movement. Both computer vision and iPPG are fed into an adaptive fusion expert of convolutional neural networks (CNN) architecture, and this boosts the accuracy at targeting of TCM radial artery at wrist. A coherence weight of their contributions was calculated and reflected the adaptation of the CNN to distance varying.