Textual Description of Dynamic Behaviors Based on Attention Mechanism

In the task of text description of physical behavior, the accuracy of conventional text description algorithms is relatively low due to the complexity of physical movements and complex background interference. For that, the paper designs a dynamic text description algorithm, SiAG, based on the attention mechanism. SiAG is mainly composed of the pose estimation module Sia and the temporal information module GRU. Sia obtains not only the target's motion features but also the background information from consecutive single-frame images. Then, the acquired motion information and background information will generate dynamic text information via the GRU module. Finally, SiAG is compared with multiple SOTAs models in the Hi-Eve dataset and ActivityNet-Caption dataset. After sufficient quantitative analysis and effect demonstration, the superiority of the SiAG model in the task of text description of physical behavior is demonstrated.