An Algorithm for Integrating Multimodal Similar Text Resources Based on Large Language Models

In the context of massive data, multimodal text resources suffer from issues such as disambiguation, making them difficult to process and utilize efficiently and accurately. This leads to reduced efficiency and recall rates in tasks such as information retrieval and text classification. To address The issue, this paper proposes an algorithm for integrating multimodal similar text resources based on large language models. Firstly, it combines the detection results of image regions and text regions from multimodal fusion to establish a comprehensive bag-of-words model, achieving document fragment annotation based on multimodal fusion of images and text. Then, a large language model combining Doc2vec and CNN is used for multimodal text representation. This involves vectorizing the annotated text data, extracting text features, and reducing dimensionality. CNN is utilized to extract deep semantic features of the text and perform feature dimensionality reduction, enabling multimodal text feature extraction. Finally, based on the multimodal text features, keyword weight indicators are determined, and accurate text similarity measurement values are calculated by combining the semantic information of known document fragments. Based on the calculated text similarity and keyword weight indicators, the k-means clustering method is applied to group similar document fragments, effectively integrating multimodal similar text resources. Experimental results demonstrate that the proposed method achieves high accuracy in multimodal text similarity classification and high recall rates in the integration of similar text resources, indicating its feasibility in multimodal data processing.