Obstacle Detection Based on the Fusion of Telephoto and Wide-Angle Images

With the rapid development of contemporary society, rail transit plays an increasingly important role in our daily lives, and higher requirements are put forward for safety and intelligence. The current object detection methods only use a single telephoto or short-focus camera for image acquisition, which cannot meet the requirements of long-distance and wide-angle detection. In response to the specific needs of object detection in rail transit application scenarios, this paper proposes a Simulated Annealing-based Normalized Cross-correlation Matching Algorithm to fuse the images of telephoto and short-focus cameras to improve the range and distance of obstacle detection. In view of the fact that most current object detection models adopt a single-input-single-output structure, in order to effectively combine the fused images and the short-focus images that provide supplementary information, this paper proposes a Dual-Input Fusion Transformer YOLO (DIFT-YOLO) network model. Finally, in order to verify the effectiveness of the proposed model, it was rigorously tested using the image data set collected on-site. The results show that under the same object detection network model, the image detection accuracy after the SA-NCC matching and fusion is 1.9% higher than that of the original short-focus image and 1.8% higher than that of the image after NCC matching. Compared with the single-input merged image model, the DIFT-YOLO model improves the mAP by 7.3%.