Dual-branch multi-modal convergence network for crater detection using Chang’e image

• A multi-modal fusion network was used for crater detection. • The multi-modal data were from the Chang’e lunar exploration mission of China. • Craters in diameters of hundreds of meters supplement the existing inventory. • Our method is effective in geomorphological feature detection on rocky planets. Knowledge about the impact craters on rocky planets is crucial for understanding the evolutionary history of the universe. Compared to traditional visual interpretation, deep learning approaches have improved the efficiency of crater detection. However, single-source data and divergent data quality limit the accuracy of crater detection. In this study, we focus on valuable features in multi-modal remote sensing data from Chang’e lunar exploration mission and propose an Attention-based Dual-branch Segmentation Network (ADSNet). First, we use ADSNet to extract the multi-modal features via a dual-branch encoder. Second, we introduce a novel attention for data fusion where the features from the auxiliary modality are weighted by a scoring function and then being fused with those from the primary modality. After fusion, the features are transferred to the decoder through skip connection. Lastly, high-accuracy crater detection is achieved based on the learned multi-modal data features through semantic segmentation. Our results demonstrate that ADSNet outperforms other baseline models in many metrics such as IoU and F1 score. ADSNet is an effective approach to leverage multi-modal remote sensing data in geomorphological feature detection on rocky planets in general.