Deep-feature-based visual localization with Google street view

Accurate visual localization in urban canyons and remote areas with unstable GNSS signals presents significant challenges. This study introduces a deep feature-based workflow that utilizes geotagged Google Street View (GSV) data for efficient and precise localization. GSV panoramas are projected into multiple perspective images and converted into deep features using a pre-trained neural network, forming a reference database. A perspective query image undergoes the same transformation, and the GSV panorama with the highest cosine similarity is identified as the best match. A learning-based tie points extraction and matching method further validates the identified GSV, followed by bundle adjustment for final localization. Additionally, a semantic segmentation model filters out query images with low localization confidence based on the presence of permanent objects. Evaluations conducted using 36 grayscale images from three stations and 2879 GSV panoramas across a 3.67 km² area achieved localization accuracy within 2 meters. The entire process takes approximately 10 seconds per image, demonstrating high accuracy and robustness in complex urban environments.