Multimodal Recommendation Systems in the LLM Era: A Survey of Feature Representation and Fusion Methods

The surge in multimodal data such as text, images, and sometimes audio, has led to a growing interest in recommendation systems capable of integrating these diverse modalities for enhanced personalization. Multimodal recommendation systems leverage such rich data to better understand user preferences and provide highly tailored recommendations. Traditional approaches employ a variety of techniques, including feature alignment, modality-specific graphs, attention mechanisms, and advanced fusion strategies, to effectively combine multimodal information. Recently, the integration of large language models (LLMs) into multimodal recommenders has transformed the field, enabling advanced preference summarization, context-aware fusion and even personalized content generation. This survey provides a comprehensive overview of both the traditional and LLM-based multimodal recommendation systems, highlighting their architectures, key methodologies, datasets, evaluation benchmarks, and real-world applications. Furthermore, we explore the challenges and opportunities posed by the fusion and processing of multi-modal data by these recommender systems.

Authors

Institutions

Publication Details

Published
2024-12-06
DOI
https://doi.org/10.1109/aeis65978.2024.00021
Primary Topic
Web Data Mining and Analysis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Multimodal Recommendation Systems in the LLM Era: A Survey of Feature Representation and Fusion Methods

Bhaskarjyoti Das, M. Varun, Ganesh Vaidyanathan K
Web Data Mining and Analysis
article

Multimodal Recommendation Systems in the LLM Era: A Survey of Feature Representation and Fusion Methods

Bhaskarjyoti Das, M. Varun, Ganesh Vaidyanathan K
article en

Abstract

The surge in multimodal data such as text, images, and sometimes audio, has led to a growing interest in recommendation systems capable of integrating these diverse modalities for enhanced personalization. Multimodal recommendation systems leverage such rich data to better understand user preferences and provide highly tailored recommendations. Traditional approaches employ a variety of techniques, including feature alignment, modality-specific graphs, attention mechanisms, and advanced fusion strategies, to effectively combine multimodal information. Recently, the integration of large language models (LLMs) into multimodal recommenders has transformed the field, enabling advanced preference summarization, context-aware fusion and even personalized content generation. This survey provides a comprehensive overview of both the traditional and LLM-based multimodal recommendation systems, highlighting their architectures, key methodologies, datasets, evaluation benchmarks, and real-world applications. Furthermore, we explore the challenges and opportunities posed by the fusion and processing of multi-modal data by these recommender systems.

PES University (IN)
Openalex Percentile: Top 51%
Web Data Mining and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Multimodal Recommendation Systems in the LLM Era: A Survey of Feature Representation and Fusion Methods — Bhaskarjyoti Das, M. Varun, et al. · (2024) | TGRS Research Map | TGRS