Human pose transfer based on gated convolution and multi-scale attention mechanism

Given the source image of a person and a different target pose, the goal of human pose transfer is to synthesize a new image that maintain the appearance details of the person under the target pose. Although attention mechanism have been widely utilized under other networks, the process of feature deformation based on attention mechanism will destroy the relative position of the original pixels, and the maintenance of the appearance details presents considerable difficulties. Aiming at this issue, this paper proposes an improved model for human pose transfer, which introduced gated convolution to predict attention maps at different scales and suppress the irrelevant region to achieve more accurate feature deformation. In this model, we first extract the texture features of the source image along with the dependency between the source pose and the target pose respectively to prepare for further feature deformation. Then, the pose features are fed to the gated convolution module to extract multi-scale attention maps, providing more accurate guidance for feature deformation. Finally, we use attention maps to gradually transform the texture features on multiple scales to complete the pose transfer process, in order to ensure the comprehensive application of the feature information at different scales. Qualitative and quantitative evaluations reveal the competitive performance of our model, achieving lower FID in DeepFashion dataset and LPIPS in Market-1501 dataset compared to existing methods.