Gaussian Sliding Window Smoothing for Human Action Segmentation

Segmenting human activities is a crucial problem in computer vision that requires accurate spatiotemporal representation of video in order to annotate human movements. Many action segmentation models have shown good performance in classifying multi-action videos and are widely used in many fields. However, most models still have over-segmentation errors, which limit the improvement of experimental results. We propose a new action segmentation framework, that completes the video segmentation task via the modules for action segmentation and feature extraction, and then punishes the over-segmentation errors through the gaussian sliding window smoothing loss function we propose. The action segmentation framework can integrate different feature extraction methods and action segmentation methods, and is extensible, which is convenient for comparison between different methods. The gaussian sliding window smoothing loss function reduces the problem of over-segmentation by using a method to punish large pieces of errors in the initial segmentation. We test our approach on two datasets. Experimental results show that our proposed action segmentation framework and loss function effectively improve the performance of video action segmentation.