Scene Graph Generation Based on Depth Information and Feature Enhancement

The objective of scene graph generation is to structure complex visual scenes into easily comprehensible graphical representations that facilitate a range of downstream visual tasks. However, existing work still exhibits the following shortcomings: firstly, an exclusive focus on the visual and semantic features of objects, with no consideration of depth information, which is also crucial for relationship prediction; secondly, the fusion of diverse modal features results in an excess of redundant object features and a lack of expressiveness; and lastly, there is a failure to mine contextual information between relationship pairs. In light of the aforementioned shortcomings, this paper proposes a scene graph generation method based on depth information and feature enhancement. This method enriches the depth geometric information of the object by mining depth features through the depth image and optimises the feature fusion process by using the Mamba algorithm, which removes the redundant information in multimodal features, enhancing their expressiveness. Additionally, a relationship pair feature enhancement method is proposed, which employs the Convolutional Block Attention Module (CBAM) to finely enhance the features of relationship pairs, providing a richer input for subsequent relationship inference. The efficacy of the proposed method was demonstrated through experimental validation on the Visual Genome (VG) dataset, wherein our method exhibited enhanced performance in the evaluation metrics of mR@20, mR@50, and mR@100. This evidence substantiates the effectiveness and superiority of the proposed method. The research presented in this paper not only advances the development of scene graph generation but also offers novel insights that can inform research in related fields.