Simulation Study: Trajectory Tracking Control of Parafoil System Based on Deep Reinforcement Learning

The parafoil system is nonlinear and complex with a large time delay. This makes it challenging for traditional control methods to control the parafoil system effectively. However, the Markov property of reinforcement learning offers a new possibility for controlling the parafoil system. Therefore, this paper employs the deep reinforcement learning (DRL) method to train a neural network controller for controlling the parafoil system, based on a modified deterministic version of the distributional soft actor-critic with three refinements (DSACT) algorithm and it is named MC-DSAC-T. The controller of the parafoil system is denoted as a multilayer perceptron (MLP) and the objective function of the policy introduces cumulative discounted rewards of a single episode to improve the stability of the iterative update of the policy, a Monte Carlo (MC) thinking. In addition, wind disturbances are introduced during training to enhance the robustness of the neural network controller. First, a nine-degree-of-freedom (nine-DOF) dynamic model of the parafoil system is developed. Secondly, the network structure of the MC-DSAC-T algorithm and the process of updating the network using sampling data were introduced. Finally, the control effects of the neural network controller trained by the proposed method were compared with those of the proportion integration differentiation (PID) controller in a wind environment. While tracking 100 randomly selected trajectory segments, the results show that the neural network controller is superior to the PID controller in distance control accuracy, which proves that the neural network controller can control the parafoil system to perform the tracking task and verify the effectiveness of the proposed method.