ToGCN-LLM: Tri-optimization Graph Convolutional Network with LLM for Group Activity Recognition

Graph networks face substantial challenges in handing large-scale graph-structured data. Although graph convolutional networks (GCNs) have been widely applied to group activity recognition, they still struggle with high graph structure complexity and semantic gap, especially in complex scenarios. To address these limitations, we harness large language models (LLMs) and multimodal learning to enhance semantic understanding as well as bridge the gap between graph structures and contextual meanings. For the simultaneous optimization of model efficiency and recognition accuracy in group activities, we propose a tri-optimization graph convolutional network with LLM (ToGCN-LLM) from the perspective of graph structure knowledge distillation. (1) To tackle high complexity of teacher network in GCN, we adopt a sparsification strategy to prune irrelevant edges and nodes, reducing computational overhead and enhancing training efficiency. (2) To mitigate information redundancy during knowledge distillation, we design a hierarchical optimization module combined with a hierarchical sampling mechanism, exploiting graph hierarchical structure and adjacency relationships to improve knowledge transfer efficiency. (3) Considering the student network’s varying learning performance across training stages and the limitations of fixed learning strategies, we introduce a dynamic adaptive weight decay mechanism to achieve fine-grained convergence under different gradient updates, thereby boosting overall recognition accuracy. (4) We use the Qwen LLM to extract text description tokens, which are fused with the student GCN’s last-layer features to enable multi-model, multi-optimization learning for group activity recognition. Six experiments on CAD, CAED and BJUT-GAD datasets demonstrate that our ToGCN-LLM achieves competitive MPCA scores of 94.89%, 93.61% and 95.77%, respectively.