Multi-Source Visual Fusion for Face-Guard Pose Estimation and Shearer Cutting-Interference Warning in Fully Mechanized Coal Mining Faces

Fully mechanized coal faces require robust sensing of hydraulic-support face-guard pose and its clearance to the shearer drum under dust, mist, and low-light conditions. This paper presents a multi-source visual fusion framework that integrates an RGB camera, an infrared camera, and a 32-line LiDAR. The system performs modality-specific denoising, distortion correction, time synchronization, spatial registration, and adaptive feature fusion before estimating face-guard 2D boxes, 3D keypoints, and 6DoF pose. RGB and infrared features are extracted with Swin Transformer backbones, LiDAR features are encoded in BEV space, and the fused representation is refined by an axial cross-modal fusion block. The recovered pose is converted into a 3D envelope, and the minimum distance to the shearer drum is mapped to a four-level warning policy. Experiments on 2400 synchronized multimodal frames from a simulated fully mechanized mining face show 98.5% [email protected] for tri-modal detection, an approximately 10 Hz integrated refresh rate, a 3D keypoint localization MAE of 1.40 cm, and a maximum environmental-condition clearance error of 2.2 cm under combined dust-and-mist disturbance. These results demonstrate the proposed method has potential application value in the field of intelligent mines.