Cross-modal media art content generation and fusion technology under the framework of deep learning

The evolution of digital media art necessitates methodologies capable of producing high-quality cross-modal content; nevertheless, the intrinsic disparities between modalities, such as text and image, hinder semantic consistency and visual fidelity in conventional methods. To address these problems, this research provides a deep learning (DL)-based system for cross-modal media art content generation and fusion, which is intended to produce semantically accurate, visually appealing, and fully detailed multimedia art material. The cross-modal media art dataset, comprising 2,500 text samples for media art generation, was gathered from an open-source platform. Pre-processing included Min–Max Normalization, Bilinear Interpolation, and Gaussian Noise Injection, Wordpiece Tokenization and lemmization are included. Textual descriptions were encoded using Bidirectional Encoder Representations from Transformers (BERT) to produce feature vectors that captured semantic information at the word and sentence levels. Feature level fusion method was used to improve Semantic alignment and visual quality. The Squirrel Search Based Self-Attention Generative Network (3SAGNet) integrates the Squirrel Search Algorithm (SSA) with Self-Attention Generative Adversarial Network (3SAGAN) for cross-modal media art generation. SAGAN generates initial visuals from BERT-encoded text features by using self-attention to align image details with textual semantics. SSA optimizes network parameters, enhancing semantic consistency, visual quality, and diversity in the generated art and design images. The experiment were implemented by Python 3.10. The proposed 3SAGNet attained an Accuracy of 95%, F1-score of 95.1%, Precision of 94%, average PSNR of 31.8, average SSIM of 0.91, Image Clarity Rating of 8.7, and an Inference Speed of 0.042 s, while compared to the existing DL methods.