Dynamic Attention-Guided Video Generation from Text with Multi-Scale Synthesis and LoRA Optimization
This paper introduces DAG-VSNet, a new architecture for high-quality text-to-video generation. The model uses a Dynamic Text Encoder (DTE), Multi-Scale Video Generator (MS-VG), Dynamic Attention Map (DAM), and LoRA-Enhanced Video Generator (LVG). The DTE captures temporal patterns, and the DAM helps focus on important text features. The MS-VG improves video details at different scales, and the LVG fine-tunes pre-trained weights with less computation. Experiments show DAG-VSNet works better than baseline models in metrics like Inception Score (IS), Fréchet Inception Distance (FID), Temporal Consistency Score (TCS), and Text-Video Alignment Score (TVAS). Ablation studies show that LoRA, dynamic attention, and multi-scale methods improve performance. This work provides a useful framework for future research in text-driven video generation.
Authors
- Jiangtian Pan
- Han Wu (ORCID: https://orcid.org/0000-0002-9113-9774)
- Shicheng Zhou (ORCID: https://orcid.org/0000-0001-5862-1991)
- Ruijie Zhu (ORCID: https://orcid.org/0000-0002-7196-6860)
- Yuan Gao (ORCID: https://orcid.org/0000-0002-0181-0991)
- Hanlu Zhang (ORCID: https://orcid.org/0009-0007-2217-9186)
Institutions
- Stevens Institute of Technology (US)
- University of Minnesota (US)
- University of Wisconsin–Madison (US)
- Wuhan University (CN)
Publication Details
- Published
- 2025-03-21
- DOI
- https://doi.org/10.1109/iscait64916.2025.11010506
- Citations
- 1
- Primary Topic
- Video Analysis and Summarization
- Type
- article
- Field-Weighted Citation Impact
- 1.28