Enhancing point cloud feature extraction for effective robot perception

Point cloud feature extraction and the rotation matrix prediction are fundamental tasks in robot perception and 3D computer vision, with critical applications in robot pose estimation, object recognition, and manipulation based on LiDAR, RGB-D, or regular RGB cameras mounted on robots. However, existing methods typically address these two problems separately, often overlooking the intrinsic relationship between them. In this paper, we propose an innovative learning framework that jointly considers rotation invariance and the rotation matrix prediction to enhance point cloud feature extraction. Specifically, we use two parallel branches to extract features from the point clouds. One branch predicts the rotation matrix based on different feature representations. The other branch ensures the consistency of global features between the rotated point clouds for downstream tasks. By balancing the variability and invariance of the features, our approach further improves the robustness and accuracy of downstream tasks. Additionally, we introduce a multi-scale feature extraction module (MSFE), which better captures the local features of the point clouds. We also introduce an attention-based global feature aggregation (AGFA) module, which enhances the capture of global features, leading to improved overall performance. Our method is not only effective but also lightweight. It has relatively small parameters and low computational requirements, which are well-suited for deployment on mobile devices. It has the potential to significantly enhance robot capabilities in object recognition, perception, and navigation tasks, especially in dynamic and unstructured environments. • A novel framework joining rotation invariance and rotation matrix prediction. • Achieved feature decoupling to balance structured variability and invariance. • MSFE module extracts multi-level local features for complex point cloud shapes. • AGFA module utilizes attention to aggregate robust and precise global features. • Achieved SOTA performance in 3D tasks with a lightweight, robot-ready design.