Musipainter: A music-conditioned generative architecture for artistic image synthesis

Generative art is a challenging area of research in deep generative modeling. Exploring AI’s role in human–machine co-creative processes requires understanding machine learning’s potential in the arts. Building on this premise, this paper presents Musipainter, a cross-modal generative framework adapted to create artistic images that are historically and stylistically aligned with 30-second musical inputs, with a focus on creative and semantic coherence. To support this goal, we introduce Museart, a dataset designed explicitly for this research, and GIILS, a creativity-oriented metric that enables us to assess both artistic-semantic consistency and diversity in the generated outputs. The results indicate that Musipainter, supported by the Museart dataset and the exploratory GIILS metric, can offer a foundation for further research on AI’s role in artistic generation, while also highlighting the need for systematic validation and future refinements. • Generation of artistic images semantically (culturally, historically, emotionally) coherent with music inputs. • A newly customized dataset consisting of music and images semantically related. • Definition of a creativity metric that evaluates semantic similarities among images generated from inputs having the same label.