Multimodal Manga Tag Estimation Using Image, Text, and Emotion Features

Manga conveys its content through multiple modalities, including page images, dialogue, and emotional progression throughout a story. This study investigates how image, text, and emotion information contribute to volume-level manga tag estimation and whether combining these modalities improves estimation performance. We estimate three tag types–publication decade, target readership, and genre–for 109 volumes in the Manga109 dataset. We extract image features from two-page spread images using CLIP, derive text features from the lines contained in each spread, and obtain emotion features from valence–arousal emotion arcs generated by a large multimodal model. We examine logistic regression, random forest, and support vector machine classifiers, and combine their class probabilities using weighted late fusion. The results show that image information is the most effective single modality for publication decade and target readership, whereas text information is the most effective for genre. Multimodal fusion did not improve publication-decade or target-readership estimation, whereas combining text and emotion features improved genre estimation. A genre-level analysis further shows that the most effective modality varies across genres: text is effective for many content-based genres, image information contributes to visually distinctive genres, and emotion information complements text for several narrative genres. These results show the importance of selecting and combining modalities based on tag type and individual genre.