Cross-domain Action Decision Algorithm for Unmanned Vehicles Based on Reinforcement Learning

The cross-domain action decision-making problem for autonomous vehicles over two periods is a partially observable Markov decision problem. Algorithms struggle to achieve stable results in unknown environments. Existing autonomous vehicle control methods have limited application to cross-domain scenarios for amphibious vehicles and find it challenging to handle unknown scenarios. To address these issues, we incorporated the concept of a world model and designed an environment prediction model tailored for cross-domain scenarios of amphibious autonomous vehicles. By leveraging collected historical information, we predict the environmental information that might be observed in the next moment, enhancing the model’s adaptability to unknown environments. We then used an LSTM network to boost the model’s ability to learn from sequential historical information, thereby improving its prediction accuracy. The PPO algorithm, known for its strong learning capabilities, was employed as the decision-making module of the system. Combined with the modules for collecting historical information and predicting future states, this formed the WLPPO algorithm. Through ablation and comparison experiments, we demonstrated that the WLPPO algorithm exhibits superior convergence ability, stability, and generalization capability. Additionally, the prediction model contributes to the algorithm’s stability and convergence ability.