Trajectory Planning for Reusable Launch Vehicle With Multisolution-Based Maximum Entropy Reinforcement Learning

Online trajectory planning is very important to the reusable launch vehicle (RLV) in emergency return cases. However, computing the optimal trajectory instantaneously is challenging, particularly when there is complicated nonlinear coupling between strict constraints, load-thermal factors, and dynamics. To cope with this problem, this paper begins with a return profile that easily satisfies strict constraints. Moreover, to mitigate the nonlinear coupling and computational cost for online trajectory planning, we propose two types of analytical models for return trajectories based on the quasi-equilibrium glide condition (QEGC) in the time-domain and another type of analytical model under phugoid oscillation condition in the Laplace frequency domain. Thus, the proposed models depict the return trajectory in both the time and frequency domains, breaking the limitations on return scenario. By contrast, most of existing algorithms are limited to either QEGC or jumping flight. The proposed analytical models also take consideration of different aerodynamic expressions and assumptions, which makes our method adaptive to various return scenarios, flight states, and computational efficiency requirements. Moreover, we devise the maximum entropy strategy and adaptive noises for the reinforcement learning agent training and accomplish online trajectory planning by using the trained optimal agent. In the long period of planning, the agent determines the most appropriate return profile and type of analytical model based on the real-time return status. In the short period of planning, the agent computes the three-dimensional trajectory based on the selected profile and model via instantaneous decision. The numerical simulation validates the method performances: the training exhibits robust and the agent has adaptively selected the analytical model with the best accuracy.