A Multimodal Recommendation Algorithm Integrating Project Text and Image Information

Traditional collaborative filtering recommendation algorithms depend exclusively on user-item interaction data, which provides limited information and makes it challenging to effectively address the data sparsity issue. In this work, we propose a multimodal recommendation model (FVMF) that integrates project text and image information. First, we design a multi-level image feature extraction module that captures image features through two approaches: one utilizes a pre-trained VGG16 model to extract global features, while the other proposes a basic image feature extraction network to extract structural features—such as color and shape—which are then fused to generate a unified image representation. Subsequently, extract textual features from the project, which are then amalgamated with the image features. Finally, the fused multimodal features are combined with the rating interaction matrix in a weighted manner to recommend items. Experimental results on two real datasets demonstrate that the proposed model effectively mitigates the limitations of traditional collaborative filtering methods caused by data sparsity.