Dual Fusion Robustness Framework with Re-weighting Attention based on Multi-source Data Fusion for Space Non-cooperative Target

It is an important aspect to fully leverage the multi-source heterogeneous data for deep learning task. Prior works mainly rely on characteristic understanding from a single sensor such as camera or LiDAR, limiting their robustness in the event that a key sensor is severely blocked. In this paper, we present a classification and pose estimation network by dual fusion multi-source heterogeneous data of point clouds and images. Instead of solely relying on single mature 2D or 3D object detectors, it realizes the task of exact classification by a robustness dual fusion network and a re-weighting attention non-linear fusion function. Moreover, the effectiveness of integrating two- and three-dimensional data from disparate data sources is proved mathematically and experimentally. Then, we introduce a feature-based 6D pose estimator, which precisely estimates rigid-body transformation from two continuous frames of input point clouds with noise. On the ModelNet dataset and a real-world vision rotating platform dataset, our proposed method also gets a better pose estimation effect of non-cooperative target.