Mediafusionnet for automated digital media content recognition using multimodal deep learning
With the explosive growth of short-video platforms, live streaming, and social media, automatically recognizing and classifying digital media content with high efficiency and accuracy has become a core task in information retrieval, content moderation, and recommendation systems. However, existing methods predominantly rely on single-modality features, which fail to capture the rich semantic information embedded in media content and lack adaptive modeling of modality-contribution differences across diverse content types, resulting in significantly limited recognition accuracy in heterogeneous content scenarios. To address these challenges, this paper proposes MediaFusionNet , a novel multimodal deep learning model that achieves hierarchical fine-grained semantic interaction and content-adaptive deep fusion of textual, acoustic, and visual modalities within a unified framework. The proposed Hierarchical Cross-Modal Attention (HCMA) module performs pairwise multi-head attention computation across all three modality pairs and aggregates cross-modal signals via learnable weights, capturing fine-grained cross-modal interactions followed by target-specific multi-source semantic aggregation. The Adaptive Gated Fusion Network (AGFN) dynamically allocates per-modality fusion weights conditioned on the semantic characteristics of each input sample via Sigmoid scoring and Softmax temperature normalization, supplemented by a GELU residual bypass to compensate for information loss. We organized the Multimodal Media Content (MMC) dataset from public multimodal sources using existing labels or metadata mappings, comprising 27,640 annotated video segments across five content categories (News, Entertainment, Education, Sports, Lifestyle). Comprehensive experiments against seven baseline methods on MMC demonstrate that MediaFusionNet achieves the best performance among the compared methods across five metrics—Accuracy (87.3%), Macro-F1 (86.1%), Precision (86.7%), Recall (86.4%), and AUC (0.934)—surpassing the strongest compared baseline by 2.4 percentage points in accuracy. Ablation studies, controlled fusion-strategy comparisons, representation-transfer evaluation, computational cost analysis, and modality-weight analysis further validate the effectiveness of each module and the overall robustness of the proposed approach. This work provides a new technical pathway for multimodal media content understanding, with significant academic and practical implications.
Authors
- Long Wang (ORCID: https://orcid.org/0009-0002-5325-498X)
- Yu Jiang (ORCID: https://orcid.org/0009-0000-4503-6208)
- Hailong Li (ORCID: https://orcid.org/0009-0006-9462-6347)
Institutions
- Shandong Huayu University of Technology (CN)
Publication Details
- Journal
- Discover Artificial Intelligence
- Published
- 2026-11-20
- DOI
- https://doi.org/10.1007/s44163-026-02779-8
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00