Feature-adaptive aggregation guided multiview stereo reconstruction network

Currently, the majority of deep-learning-driven Multi-View Stereo (MVS) models rely on Feature Pyramid Networks (FPN) to extract multi-scale image features and construct a cascade architecture to restore the real depth information and 3D structure of scenes from 2D images. However, such networks often face misalignment issues between high-level and low-level features with respect to spatial positioning when processing multi-scale feature maps, which can significantly affect the model’s 3D reconstruction accuracy and its ability to capture fine details. To overcome this limitation, this study presents a new MVS reconstruction network called FAG-MVSNet. It uses a feature alignment module to adapt the spatial position of feature maps by learning the pixel offset, enabling accurate alignment of contextual features. Additionally, a spatial pooling selection module helps enhance low-level features rich in spatial details, which assists in the aggregation of contextual features. Results from the DTU dataset indicate that FAG-MVSNet achieves superior reconstruction quality compared to other approaches.