Bi-Level Routing Attention and Enhanced Spatial-Temporal Inconsistency Learning for Deep VFI Video Detection

With the maturation of Deep Learning-based Video Frame Interpolation (Deep VFI), the left spatial-temporal inconsistency in the synthesis process is greatly improved, which poses a challenge to the current VFI detector. This article presents a dual-stream identification network based on Bi-level Routing Attention and enhanced Spatial-Temporal inconsistency learning (BRA-ST) to address this challenge. Specifically, the spatial inconsistencies in Deep VFI are mainly reflected in their motion regions and moving object edges; thus, the high-pass filter is introduced to enhance them, facilitating the three-stage pyramid structure of BiFormer Blocks with bi-level routing attention in the frame-level stream to learn. To fully exploit the temporal inconsistencies in the Deep VFI video, the time-difference module in the time-level stream is superimposed with the ConvGRU to extract the temporally dependent features of continuous multiple frames. Additionally, the middle layer of the two streams interacts and aggregates with the channel attention, and then, their last layer adaptively merges from a whole and part perspective for the ultimate frame prediction. Finally, the experimental findings on a constructed dataset by the five most advanced Deep VFI methods indicate that the proposed BRA-ST achieved \(F_{\text{1Score}}\) of 99.73%, which is superior to the existing Deep VFI detectors, and further verify that the resolution of BRA-ST for different Deep VFI methods reached 78.55%. Our source codes and dataset are available at https://pan.baidu.com/s/1f05_gS0qu5G-SSIkd9F4Hw?pwd=j6t6 .