Robust geometry-adaptive transformer engine for simulation-to-real point cloud classification

Unsupervised domain adaptation for simulation-to-real (Sim2Real) point cloud classification poses significant challenges due to domain disparities between synthetic and real-world 3D data. This work introduces “Robust Geometry-Adaptive Transformer Engine (RGATE),” a novel framework that enhances state-of-the-art performance in this domain. Our approach leverages multi-scale transformer networks with geometry-aware local loss for improved local feature extraction, augmented by contrastive learning, advanced pseudo-labeling, and domain query alignment for robust global representation. Incorporating multi-modal inputs and a pre-trained backbone further enriches contextual understanding and accelerates training. Evaluated on the point cloud domain adaptation on data from 10 common categories (PointDA-10) dataset, our model achieved 89.0% accuracy and 75.23% average accuracy across six settings. Analysis of the confusion matrix and per-class metrics highlights strong performance, with targeted refinements addressing underperforming classes. Our method demonstrates scalability and applicability to real-world tasks such as autonomous driving and robotics. This work showcases the transformative potential of transformer-based architectures in bridging Sim2Real gaps in three-dimensional (3D) vision.