FusionNet: A Supervised Multi-Modal Deep Learning Framework for Enhanced Product Recommendation Using SBERT and MobileNetV2

The rapid expansion of e-commerce platforms has rendered recommender systems indispensable for alleviating information overload and facilitating product discovery. However, conventional collaborative filtering and content-based approaches are fundamentally constrained by data sparsity, the cold-start problem, and their reliance on single-modality signals, which collectively undermine recommendation quality in dynamic environments. To address these limitations, this paper presents FusionNet, a supervised multi-modal deep learning framework that jointly learns unified product representations from unstructured textual descriptions and visual imagery without requiring historical user–item interactions. Specifically, textual attributes are encoded using a Sentence-BERT model fine-tuned on the Stanford Natural Language Inference (SNLI) and Multi-Genre Natural Language Inference (MNLI) corpora to capture nuanced semantic relationships. Concurrently, visual features are extracted via a lightweight MobileNetV2 backbone and projected into a compatible embedding space. The MobileNet model is specifically designed for high-efficiency operation on smart devices and embedded systems. It achieves this by drastically reducing its computing resources compared to traditional neural networks, requiring only very low processing power (low computation) because it relies on Depthwise Separable Convolutions instead of traditional convolution. Furthermore, MobileNet occupies a very small amount of storage space (between 16 and 32 MB depending on the version used) and reduces the number of calculations, thus minimizing the load on the processor. These heterogeneous features are subsequently fused through a multilayer perceptron-based fusion network, which is explicitly supervised by product category labels to learn a discriminative shared embedding space of 256 dimensions. Recommendation retrieval is then performed via cosine similarity over this unified space. Experiments conducted on Amazon product corpus comprising 459,229 items across 112 categories demonstrate the efficacy of the proposed framework. FusionNet achieves a Precision@10 of 0.860, an NDCG@10 of 0.861, and a HitRate@10 of 0.933, indicating that over 93% of the generated recommendation lists contain at least one relevant item within the top ten positions. Furthermore, at k=7, the model attains an NDCG of 0.902 and a product coverage of 0.920, confirming its ability to surface diverse and relevant items effectively. Comprehensive ablation and robustness analyses further corroborate that the synergistic integration of semantic and visual modalities, guided by categorical supervision, significantly mitigates the cold-start challenge and enhances overall system reliability. The results establish FusionNet as a highly accurate, scalable, and deployable content-driven solution for modern e-commerce and digital content platforms.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24416743
Primary Topic
Recommender Systems and Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

FusionNet: A Supervised Multi-Modal Deep Learning Framework for Enhanced Product Recommendation Using SBERT and MobileNetV2

Sira Astour, Tharaa Melhem
Zenodo (CERN European Organization for Nuclear Research)
Recommender Systems and Techniques
article

FusionNet: A Supervised Multi-Modal Deep Learning Framework for Enhanced Product Recommendation Using SBERT and MobileNetV2

Sira Astour, Tharaa Melhem
article en

Abstract

The rapid expansion of e-commerce platforms has rendered recommender systems indispensable for alleviating information overload and facilitating product discovery. However, conventional collaborative filtering and content-based approaches are fundamentally constrained by data sparsity, the cold-start problem, and their reliance on single-modality signals, which collectively undermine recommendation quality in dynamic environments. To address these limitations, this paper presents FusionNet, a supervised multi-modal deep learning framework that jointly learns unified product representations from unstructured textual descriptions and visual imagery without requiring historical user–item interactions. Specifically, textual attributes are encoded using a Sentence-BERT model fine-tuned on the Stanford Natural Language Inference (SNLI) and Multi-Genre Natural Language Inference (MNLI) corpora to capture nuanced semantic relationships. Concurrently, visual features are extracted via a lightweight MobileNetV2 backbone and projected into a compatible embedding space. The MobileNet model is specifically designed for high-efficiency operation on smart devices and embedded systems. It achieves this by drastically reducing its computing resources compared to traditional neural networks, requiring only very low processing power (low computation) because it relies on Depthwise Separable Convolutions instead of traditional convolution. Furthermore, MobileNet occupies a very small amount of storage space (between 16 and 32 MB depending on the version used) and reduces the number of calculations, thus minimizing the load on the processor. These heterogeneous features are subsequently fused through a multilayer perceptron-based fusion network, which is explicitly supervised by product category labels to learn a discriminative shared embedding space of 256 dimensions. Recommendation retrieval is then performed via cosine similarity over this unified space. Experiments conducted on Amazon product corpus comprising 459,229 items across 112 categories demonstrate the efficacy of the proposed framework. FusionNet achieves a Precision@10 of 0.860, an NDCG@10 of 0.861, and a HitRate@10 of 0.933, indicating that over 93% of the generated recommendation lists contain at least one relevant item within the top ten positions. Furthermore, at k=7, the model attains an NDCG of 0.902 and a product coverage of 0.920, confirming its ability to surface diverse and relevant items effectively. Comprehensive ablation and robustness analyses further corroborate that the synergistic integration of semantic and visual modalities, guided by categorical supervision, significantly mitigates the cold-start challenge and enhances overall system reliability. The results establish FusionNet as a highly accurate, scalable, and deployable content-driven solution for modern e-commerce and digital content platforms.

Zenodo (CERN European Organization for Nuclear Research)
Arab International University (SY), Syrian Virtual University (SY)
Recommender Systems and Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon