The Video Feature with Diffusion Convolution For Temporal Action Proposal Generation

Graph convolutional network can be utilized to extract video features for temporal action proposal generation. Even though both temporal and semantic relations are included, their extracted features perform poorly in finding action periods on a small network. This paper introduces diffusion convolutional neural networks to enrich video features as diffusion features. This network helps to expand both temporal and semantic relations in the graph to enrich video features. In addition, the diffusion feature helps to emphasize the action proposals within the graph network and acts as a low-pass filter to smooth noisy features within the graph. In the experiment, the state-of-the-art of boundary-based method is utilized as a baseline for evaluation. Extensive experiments are evaluated on a large temporal proposal dataset, ActivityNet v1.3. Evaluation results demonstrate that the proposed method achieved a 1% average recall improvement over the baseline. By analyzing the video feature, the high feature similarity between action and background proposals helps to increase the accuracy for localizing their temporal boundaries. Then, the diffusion feature from background and action proposals has a higher difference score than its baseline video features.