Design of a Hexacopter Attitude Controller Based on Reinforcement Learning with Transfer Learning Application

A method utilizing reinforcement learning and transfer learning for hexacopter attitude control is proposed. The reinforcement learning approach employs Proximal Policy Optimization (PPO), which enables continuous output and prevents abrupt policy updates, ensuring stable learning. Transfer learning, which accelerates learning by adapting a previously trained agent to a similar environment, is applied to develop a fast and efficient hexacopter attitude controller. The research process consists of three major phases. In the first phase, a quadcopter attitude control policy is trained using reinforcement learning to achieve stable flight and maneuvering. In the second phase, the output of the pre-trained quadcopter control model is modified to facilitate its transfer to a hexacopter. In the final phase, the pre-trained model is used as the initial learning model, and transfer learning is applied to extend the quadcopter controller to a hexacopter attitude control system. The performance of the transfer-learned hexacopter controller is validated through dynamic simulations. To evaluate the effectiveness of transfer learning, a comparative study is conducted by measuring the time required for a hexacopter controller to reach full training completion using standard PPO and comparing it to the training time when transfer learning is applied. Instead of a single measurement, multiple training sessions are performed, and the average required training time is analyzed. The results demonstrate that transfer learning enables the hexacopter to achieve similar performance in a shorter amount of time compared to training from scratch using PPO.