Scaled Background Swap: Video Augmentation for Action Quality Assessment with Background Debiasing

Action quality assessment (AQA) has become crucial in video analysis, finding wide applications in various domains, such as healthcare and sports. A significant challenge faced by AQA is the background bias due to the dominance of the background in videos. Especially, the background bias tends to overshadow subtle foreground differences, which is crucial for precise action evaluation. To address the background bias issue, we propose a novel data augmentation method named Scaled Background Swap. First, the background regions between different video samples are swapped to guide models focus toward the dynamic foreground regions and mitigate its sensitivity to the background during training. Second, the video’s foreground region is upscaled to further enhance models’ attention to the critical foreground action information for AQA tasks. In particular, the proposed Scaled Background Swap method can effectively improve models’ accuracy and generalization by prioritizing foreground motion and swapping backgrounds. It can be flexibly applied with various video analysis models. Extensive experiments on AQA benchmarks demonstrate that Scaled Background Swap method achieves better performance than baselines. Specifically, the Spearman’s rank correlation on datasets AQA-7 and MTL-AQA reaches 0.8870 and 0.9526, respectively. The code is available at: https://github.com/Emy-cv/Scaled-Background Swap.