Novel Multitask Action Quality Assessment Model Based on X3D

Action quality assessment aims to evaluate the execution quality of video motions and is a challenging video comprehension task. Most current research methods focus on generating predictive scores, ignoring the full description of the action. However a single score is not enough to describe a complex action. To address this problem, this paper proposes a multi-task action quality assessment model that introduces the action process description task to generate video action process descriptions on the basis of predicted scores for a more comprehensive evaluation of action execution. To address the lack of text labels in the current action quality assessment data, the BERT model is introduced, a pre-training-fine-tuning approach is used to reduce the data requirements, and a bidirectional LSTM is used to fuse video and text information. For the score regression task, most of the current methods split the video into multiple segments, use 3D networks to extract the features of each segment, and stack them, ignoring the importance differences between the segments. To address this problem, this paper proposes an adaptive weight decision module to help the model focus on the important information, and finally generates the predicted scores through KAN network. The experimental results show that the Spearman rank correlation coefficient of the score regression model proposed in this paper achieves 0.907, which is 0.011 and 0.06 higher than the traditional C3D-AVG and MSCADC, respectively, and the text generation model proposed in this paper only needs 1–3 rounds of training, and the BLEU and ROUGE-L reach 0.28 and 0.33, which are 0.12 and 0.06 higher than MSCADC, respectively.