Reinforcement Learning Generalization for Quadrotor With Slung Load Systems Through Homogeneity Transformations

Load transportation through unmanned aerial vehicles (UAVs), such as quadrotors, has a high potential for quick deliveries to locations that are out of the reach of ground vehicles. The complexity of the pick-and-place procedure in such tasks increases if the target location does not have a clearance at the top, necessitating the use of recent learning-based controllers such as reinforcement learning (RL). This paper presents a new concept of dual-scale homogeneity, a property defined by scaled magnitudes and time in transformed coordinates that remain independent of system parameters. It demonstrates that applying transformations to achieve this property ensures consistent performance of a quadrotor with a slung load system (QSLS) despite variations in its parameters. Furthermore, it also presents an effective approach to design a parameter-dependent RL policy that homogenizes the QSLS. Unlike plain RL or gain-scheduled PID controllers which confine parameter variations within a predefined range encountered during training or tuning, the developed approach works under large parameter variations, significantly surpassing the performance of traditional controllers. The conducted experiments on load placement in a confined space, utilizing a quadrotor to manage load swing, proved the proposed synergy between the homogeneity transformations and reinforcement learning, yielding a success rate of 96% in bringing the load to its designated target with a 3D RMSE of 0.0253 m. The video that shows the experimental results along with a summary of the paper is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://youtu.be/fdC0gHXzwdE</uri>.