EMF-PAL: enhanced multi-sensor fusion algorithm for high-precision perception and localization in challenging environments

Abstract This study proposes an enhanced multi-sensor fusion perception and localization algorithm named EMF-PAL, which leverages the strengths of three distinct sensor types to achieve high-precision localization. The algorithm employs a lightweight object detection network to extract semantic information from the environment, which is combined with feature points obtained through parallel bidirectional optical flow tracking. A dual-strategy approach is introduced to identify reliable static feature points, enabling accurate pose estimation. Sensor data is dynamically updated using factor graph optimization to ensure accurate and continuous real-time localization. In the feature preprocessing module, the lightweight object detection algorithm is integrated with Shi-Tomasi corner detection to provide prior information for subsequent object recognition and filtering. Additionally, the high-precision inertial measurement unit (IMU) supplies translation and rotation data, enabling short-term, high-accuracy positioning and motion compensation, effectively reducing misclassification during rapid movements. The Global Navigation Satellite System-Real-time Kinematic provides all-weather, drift-free global localization information without the need for additional transformations, further enhancing the factor graph optimization for position updates and supplying auxiliary data to the IMU. EMF-PAL maximizes the advantages of multiple sensors, making it well-suited for complex environments. Extensive comprehensive experimental validation using real outdoor environment sequences under different environmental conditions is conducted to demonstrate the real-time accuracy of EMF-PAL and its ability to cope with complex environments. The experimental results demonstrate that the proposed method enhances localization accuracy by up to 50.3% in challenging outdoor environments compared to state-of-the-art algorithms utilizing three-sensor fusion, effectively fulfilling the localization and perception requirements of real-world applications.