Complete information extraction for monocular depth estimation using a dual framework

This paper aims to address the problem of efficient extraction of complete multi-scale information for supervised monocular depth estimation. Most of the existing depth estimation methods are based on Convolutional Neural Network (CNN). By gradually exploring the contextual and semantic features, they have achieved good results in scene depth estimation. However, with the expansion of the receptive field, global information limited by the local induction bias is gradually suppressed, resulting in the performance cannot be further improved. Recently, Transformer-based methods have been widely used to model the global correlation between features. Nevertheless, since the Transformer networks are not spatially aware enough, they usually lose local details and have no clear mechanism for reusing features when processing images. The Transformer networks perform self-attention mechanism at each location and cannot directly obtain information from other locations for features. Therefore, we propose a novel dual framework called as Transformer-CNN, which includes the Transformer-branch and the CNN-branch for monocular depth estimation. Specifically, the Transformer-branch is able to model the global contextual information and the CNN-branch can capture local spatial relationships in images. However, simply fusing these two independent branches may result in insufficient feature aggregation. To this end, we design a Parallel Feature Interaction Module (PFIM), which contains a Self-Attention Module (SAM) and a Cross-Attention Module (CAM), so as to highlight features from the Transformer-branch and the CNN-branch respectively and extract complementary information between the two branches. Meanwhile, in order to make full use of the low-level features with low quality in the scene, we propose a Low-level Information Acquisition Module (LIAM) to capture texture-related information and preserve texture details in the CNN-branch. Finally, to address the lack of multi-scale contextual information in Vision Transformer (ViT), we introduce a Wide Area Multi-scale Decoder (WAMD), which incorporates the multi-scale feature representations into the decoder part via a Wide Area Attention (WAA). Extensive experiments on benchmark datasets collected in the outdoor and indoor environments demonstrate the competitive results of the proposed method, compared with the state-of-the-art monocular depth estimation methods. • A dual Transformer-CNN framework is proposed. It combines the global context modeling ability of the Transformer-branch with the local spatial information capture ability of the CNN-branch. • A Parallel Feature Interaction Module (PFIM) is designed, and it enhances the feature extraction capabilities of different branches and promotes information exchange. • A Low-level Information Acquisition Module (LIAM) based on statistical methods is proposed to effectively capture the texture information. • A Wide Area Multiscale Decoder (WAMD) is introduced, improving the depth prediction performance of the model in complex scenes.