BSSA-net: category-level 6D object pose estimation with bi-scale and semantic-aligned feature interaction

Purpose Category-level 6D object pose estimation is essential for robotic grasp detection and manipulation. However, large distribution gaps between synthetic and real data, the limited capacity of single-branch architectures to capture multi-modal features and the high cost of manual annotations remain critical challenges. This study aims to present BSSA-Net, a novel three-branch parallel architecture to effectively address these limitations and enhance pose estimation performance in real-world scenarios. Design/Methodology/Approach BSSA-Net is a three-branch parallel architecture trained on synthetic data and designed to generalize to real datasets. It incorporates a semantic-aligned knowledge interaction to fuse RGB and point cloud features, a bi-scale information fusion mechanism to integrate temporal cues through hierarchical features and attention modules and a hybrid scope feature extraction strategy to strengthen 3D graph convolution. Each branch follows a three-stage process – primary deformation, precise deformation and pose estimation – while a novel self-supervised loss is introduced to enable knowledge transfer without requiring extensive real-world labels. Findings Experiments conducted on CAMERA25 and REAL275 across synthetic-only, self-supervised and supervised learning settings demonstrate that BSSA-Net achieves state-of-the-art performance. Cross-dataset evaluations on Wild6D further confirm its robustness and generalization, particularly in synthetic-only and self-supervised scenarios. These results highlight the effectiveness of the proposed architecture in bridging the synthetic-to-real gap. Originality/Value This work introduces a novel multi-branch framework that fuses multi-modal features and leverages self-supervised learning, reducing reliance on labeled data while improving generalization. It advances the field of category-level 6D pose estimation by providing a scalable and robust approach for robotic perception and manipulation, offering a foundation for future research and real-world applications.