Exploring the Generalizability of Geomagnetic Navigation: A Deep Reinforcement Learning Approach With Policy Distillation

The advancement in autonomous vehicles has empowered navigation and exploration in unknown environments. Geomagnetic navigation for autonomous vehicles has drawn increasing attention with its independence from the global positioning system (GPS) or inertial navigation devices. While geomagnetic navigation approaches have been extensively investigated, the generalizability of learned geomagnetic navigation strategies remains unexplored. The performance of a learned strategy can degrade outside of its source domain where the strategy is learned, due to a lack of knowledge about the geomagnetic characteristics in newly entered areas. This article explores the generalization of learned geomagnetic navigation strategies via deep reinforcement learning (DRL). Particularly, we use DRL agents to learn multiple teacher models from distributed areas that represent dispersed navigation strategies and amalgamate the teacher models for generalizability across navigation areas. We design a reward shaping mechanism in training teacher models where we integrate both potential-based and intrinsic-motivated rewards. The designed reward shaping can enhance the exploration efficiency of the DRL agent and improve the representation of the teacher models. Upon the gained teacher models, we use multiteacher policy distillation to merge the policies learned by individual teachers, leading to a navigation strategy with generalizability across navigation areas. We conduct numerical simulations, and the results demonstrate an effective transfer of the learned DRL model from a source domain to new navigation areas. Compared with existing evolutionary-based geomagnetic navigation methods, our approach provides superior performance in terms of navigation length, duration, heading deviation, and success rate in cross-domain navigation.