A Framework for Portuguese Multimodal Summarization with Fine-Grained Visual Feature Extraction

With the rapid development of information technology, multimodal summarization has become increasingly important for information retrieval and content understanding. In recent years, many researchers have focused on exploring the integration of multimodal information, proposing various methods to enhance the quality and accuracy of summaries. However, research on low-resource languages remains relatively scarce, particularly for Portuguese, primarily due to a lack of sufficient corpora and datasets. Moreover, traditional multimodal summarization methods often rely on basic feature extraction and simple additive techniques, which fail to effectively capture fine-grained features and achieve meaningful alignment between different modalities. To address this issue, we propose a multimodal summarization model (PTMS) that emphasizes fine-grained feature extraction. Our work encompasses several key aspects: first, we implement effective fusion and alignment between modalities through fine-grained feature extraction and cross-modal attention fusion; second, we have developed a new graphic-text dataset (PT-news) by crawling data from Portuguese news websites, which includes news texts, images, and graphictext summaries. Experimental results demonstrate that PTMS outperforms existing baseline methods across multiple evaluation metrics.