Deep Art Understanding: A Unified Framework for Multi-Task Art Analysis with Cross-Modal Attention

Artworks analysis is an important research content in the field of humanities, the traditional analysis method relies on the expert experience analysis to judge, have subjective will, efficiency and accuracy are poor. In recent years, with the rapid development of artificial intelligence, it has been successfully used in many fields, such as image recognition, automatic driving, intelligent translation, and so on. In this article, we attempt to utilize AI as a means of systematic critique of artwork through a novel multimodal approach. Concretely, we develop a multimodal learning framework with dual branches and introduce the textual comments and image features respectively, and in between the two branches, we design a cross-modal interleaved self-attention mechanism which is a bridge to further match and align textual comments and image features. Extensive experimental evaluation demonstrates that our method significantly outperforms other deep learning-based approaches, achieving improvements of up to 11% in Genre classification and 21% in Author Identification, thus establishing a new state-of-the-art benchmark in AI-assisted artwork analysis.