Bidirectional Fusion of Complementary Transformer-Cnn Features for Human Pose Estimation

Aiming at the problem that it is difficult to correctly predict human pose in the case of occlusion, scale change and low resolution, a human pose estimation model based on bidirectional fusion of convolutional neural network and Transformer feature is proposed.. Using the characteristics of convolutional neural network and Transformer feature learning network, the expression ability of local features is improved while strengthening the remote modeling ability of the model. The model is mainly composed of three parts ; transformer branch which focuses on the remote modeling ability of the model ; the CNN branch that mainly extracts local detail information ; a bidirectional fusion interaction module based on deformable self-attention mechanism. The bidirectional fusion interaction module uses the attention mechanism to eliminate the architectural differences between CNN and Transformer, thereby achieving complementary features and ultimately improving the performance of the model. The experimental results show that compared with the benchmark model, the proposed model improves the accuracy by 2.3 % and 0.5 % on the COCO dataset and MPII dataset, respectively.