Sign Language Video Segmentation Using Temporal Boundary Identification

Sign language segmentation focuses on identifying temporal boundaries within sign language videos.As compared to previous segmentation techniques that have depended on frame-level and phrase-level segmentation, our study emphasizes on subtitle-level segmentation, using synchronized subtitle data to facilitate temporal boundary recognition.Based on Beginning-Inside-Outside (BIO) tagging for subtitle unit delineation, we train a sequenceto-sequence (Seq2Seq) model with and without attention for subtitle boundary identification.Training on optical flow data and aligned subtitles from BOBSL and YouTube-ASL, we show that the Seq2Seq model with attention outperforms baseline models, achieving improved percentage of segments, F1 and IoU score.An additional contribution is the development of a method for subtitle temporal resolution, which automates the generation of time-stamped SubRip Subtitle (.srt) files.