Multi-Scale Context-Aware Modeling for Sign Language Translation

Sign language translation converts sign language videos into grammatically text sentences. To address the limitations of existing CNN-based methods, such as inadequate temporal feature fusion and repetitive or ungrammatical translations, we propose a model integrating multi-scale temporal features with historical context perception. Our model includes two key modules: (1) A widerange multi-granularity feature fusion module that hierarchically processes video features across different temporal scales, enhancing temporal representation; (2) A historical context perception module that uses prior attention states to guide current translations, reducing repetition and omissions. Experiments on Chinese Sign Language and How2Sign (ASL) datasets demonstrate that our method outperforms existing approaches regarding accuracy and grammaticality, thereby showing its effectiveness.