Face Drawing GAN by Channel Attention and Matrix Product Attention

Face photo-sketch synthesis tasks have been developed with Generative Adversarial Networks (GANs) based on Convolutional Neural Network (CNN) and Vision Transformer (ViT). CNN is good at capturing local features, but its locality results in blurred images and contours. ViT is good at capturing global information, but is not as good as CNN in capturing local features, and while it can prevent blurring of contours and other lines, it does not reflect fine texture. Therefore, we propose a Face Drawing GAN, which generates high-quality face sketches by capturing both local and global features. Face Drawing GAN is a CNN-based model and it incorporates Channel Attention, which functionally adjusts the weights of channels, and Matrix Product Attention (MP Attention), which weights pixels based on the similarity between the vertical and horizontal sides of images obtained by matrix product. Through the experiments, we confirmed that our proposed MP Attention assists in capturing global features and Face Drawing GAN is capable of generating face sketches that outperform conventional methods.