Vision meta art transformer-powered adaptive style transfer for diverse art patterns

Despite significant advances in artificial intelligence, art style selection still faces a significant challenge: the inability to quickly adapt to diverse and new art styles without costly retraining of the model. This limitation limits the practical applicability of existing technologies, especially for new and unknown styles. To overcome this challenge, this paper introduces a novel framework called the Visual Meta Art Transformer (VMAT). This framework leverages the power of the Visual Transformers (ViT) architecture in understanding global image relationships and a Dynamic Style Modulation approach to provide high adaptability. Unlike conventional methods that rely solely on memorizing specific styles, VMAT learns the learning process of the style itself. In this model, a joint encoder based on the Visual Transformer extracts rich content and style features from images. A Style Modulation Network (SMN) then transforms the style essence into a set of compact modular parameters. Next, an adaptive decoder using the Adaptive Instance Normalization AdaIN mechanism injects these parameters into the content features and the final image is reconstructed. For evaluation, the WikiArt and Microsoft COCO (MS-COCO) datasets were used. The results showed that our proposed method not only performs successfully in style transfer, but also outperforms existing methods in simultaneously preserving content and style features. In addition, the model was able to perform creatively and effectively in style transfer for unseen examples and styles during the training process, achieving a score of 8.37 in style fidelity score and 7.91 in content fidelity score.