Multi modal fusion human-machine interactive nursing bed system

The present poor experience of using unimodal brain-computer interfaces is caused by their low accuracy and poor generalization ability. In this paper, we propose a novel system (BCI-GP) that integrates head posture, eye tracking and brain-computer interface. TCN network is selected for feature extraction of BCI signals, and the accuracy of the model as well as its generalization ability is improved by multi-domain feature fusion. Eye tracking is realized based on face feature points. The EfficiNetV2 network structure is selected for head posture, while multitasking is used to convert five classifications into two and three classifications, thus improving the classification accuracy and at the same time improving the generalization ability and robustness of the model. Adaptive weight fusion is used at the decision layer to achieve the BCI-GP nursing bed control. Six experimenters participated in the on-line experiments, and the experimental results show that, under the time window of 1s, the BCI-GP average control accuracy is as high as 95.17 1.69%, and the average ITR is as high as 98.87±5.88 bit/min; compared with SSVEP-BCI unimodal and GP unimodal, the average control accuracy is improved by 13.66% and 1.46%, and the average ITR is improved by 37.2 bit/s and 5.97 bit/s; BCI-GP system is more reliable.