Optical flow estimation based on global cross information and dynamic encoder–dynamic decoder

Abstract To solve the problem that the lack of a global perspective leads to local misestimation and overall structural dislocation when optical flow estimates large-scale motion and complex scenes, this paper proposes an optical flow estimation based on global cross information and dynamic encoder–dynamic decoder. The network architecture is improved stage by stage according to the streaming direction of the encoder and decoder data streams. For the encoder, relative and absolute position coding are adopted to construct a mixed coding network, and the learnable weights are used to dynamically adjust the mixed position coding information to enrich the global and local position information. For the enhancer layer after the encoder, the contextual information of the input sequence is captured through cross attention and a feed-forward neural network to construct a global cross information attention network, which acquires the global perspective under the premise of adapting to large-scale motion. For the decoder, bilinear interpolation and deformable convolution are combined to construct a dynamic anisotropic upsampling module, and dynamic anisotropic upsampling of the input feature maps is realized by adjusting the offsets of the sampling points to enhance the ability to process high-resolution details and complex motion boundaries. Finally, the performance of the proposed method is evaluated in the endpoint error value, estimation time and number of parameters. The experimental results show that compared with the RAFT benchmark network, the model in this paper preserves the global features and edge detail information of large-scale motion without significantly increasing the number of model parameters.