FD-GCN: Feedback Directed Graph Convolutional Network for skeleton-based action recognition

Graph Convolutional Network (GCN) has achieved remarkable result in skeleton-based action recognition. In GCNs, multi-order information has shown notable improvement for recognition and the graph topology, which is the key to fusing and extracting representative features. However, the GCN-based methods still face the following problems: (1) Nodes will have over-smooth problems in deep and complex networks. (2) Lack of efficient methods to fuse data streams of different modalities. In this paper, we proposed a novel data-fusing method, Feedback Directed Graph Convolution (FD-GC), to dynamically construct diverse correlation matrices and effectively aggregate both joint and bone features in different hierarchical update state and utilize them as feedback loops to participate in aggregation respectively for both streams. Our methods significantly reduce the difficulty of modeling multi-streams features at a small parameter cost. Furthermore, the experimental results indicate FD-GC alleviates the over-smooth effect via the feedback mechanism, constructing stronger representation capabilities of fine-grained actions, and performs as well as most skeletal motion recognition algorithms on two large public datasets NTU RGB+D 60, NTU RGB+D 120 and Northwestern-UCLA. • FD-GC verified the effectiveness of contribution of high-level feedback feature information. • FD-GCN reduce the influence of over-smooth on aggregation at a small parameter cost. • FD-GCN verify and visualize the representation of motion information by directed topology. • FD-GCN performs as well as most skeletal recognition algorithms on three large public datasets.