One-shot handwriting imitation via self-supervised cross spatial transformer networks

Generating stylized characters has been a focus of research for a long time. However, current font-led algorithms in the field of handwriting generation often perform suboptimally due to severe geometric distortion among handwriting samples. Additionally, the non-Euclidean characteristic of hidden embedding is identified as another key issue in our research. To address shape distortion, we propose a Self-Supervised Style Learning (SSL) method, aiming to increase the skeleton similarity between the output image and target. Furthermore, to enhance the Euclidean spatial distributions of the generator, this paper introduces a joint spatial transformer by refining the fusion process of content and style information using an attention mechanism . Moreover, the training process of the current GAN is reformed to meet the requirements of SSL. As a result, the proposed method enriches the calligraphy diversity of stylized handwriting generation compared to existing font generation algorithms. Evaluation of the proposed method using a public Chinese handwriting database demonstrates that our approach generates handwriting images with superior quality, stroke correctness, and calligraphy similarity when compared to other prevailing font imitation methods. • One-shot input sample for imitating writer’s handwriting calligraphy. • High image quality, stroke skeleton similarity and style diversity. • Exploration of non-Euclidean characteristic of networks. • Cross spatial transformer and self-supervised training.