Research on Visual Localization of the Pallet Relative to the AGV Based on Image Matching

In factory environments, although laser radar is widely used for AGV navigation, when pallets are being forked, due to the position error of the pallets, visual localization of the pallets relative to the AGV is still required when the AGV navigates to the vicinity of the pallets. In this paper, the visual localization of the pallet relative to the AGV is carried out based on the extraction and matching of ORB feature points and the pose optimization based on the Gauss-Newton method. The main research includes the following aspects: first, the AGV and the pallet are adjusted to the standard position in advance, the standard RGB image and depth image are taken, and the realtime acquired image in operation and standard RGB image are used for ORB feature points extraction. Next, brute-force matching was used to establish the initial matching among the extracted feature points, and the coarse and fine matching of feature points was carried out according to the Grid-based Motion Statistics (GMS) algorithm and the RANSAC algorithm to remove the mismatched points. Finally, the 3D positions of feature points in the standard RGB image and the feature points in the real-time image are paired to form 3D-2D point pairs. The current precise pose of the pallet relative to the AGV is determined by the Gauss-Newton method based on the reprojection error minimization. Experimental results suggest that the method proposed in this paper attains good localization accuracy.