RGPFormer: Input-Adaptive Geometric Bias Calibration for RGB-D Semantic Segmentation

Abstract Purpose: RGB-D semantic segmentation uses depth information to reduce appearance ambiguity. However, geometric prior weights shared across inputs cannot adjust the contributions of spatial and depth cues to individual scenes. Pairwise depth differences also do not explicitly represent local depth variation. This work introduces a mechanism that calibrates geometric biases for each input using its relative depth distribution and local depth cues. Methods: We propose the Reliability-aware Geometry-Prior Transformer (RGPFormer). Adaptive Geometry Prior Gating (AGPG) uses each sample's relative depth distribution to generate separate overall scaling factors for the spatial and depth biases in each attention head at each stage. Local Depth Reliability Modulation (LDRM) derives a scaling factor for each position in every attention head from normalized depth, local depth standard deviation, and depth gradient. For each position pair, the product of its two endpoint factors modulates the corresponding depth bias. The full configuration applies asymmetric residual scaling to AGPG in the final encoder stage and adds the calibrated geometric bias to the scores between queries and keys before Softmax. Results: AGPG and LDRM add 0.033M parameters. The evaluated configuration based on DFormerv2-S, denoted RGPFormer-S, achieved mean intersection over union (mIoU) scores of 58.25% on NYU Depth V2 and 52.16% on SUN RGB-D. Compared with the matched baseline A0, these scores were 0.94 and 0.49 percentage points higher, respectively. Conclusion: RGPFormer uses the relative depth distribution and local depth cues of each input to adapt attention with low parameter overhead.