Reinforcement learning-based optimal pitch-angle generation for energy-efficient tilt-rotor quadrotors

• A new control strategy improves energy efficiency under dynamic wind conditions • Aerodynamic coefficients derived as functions of angle of attack via CFD analysis • Reinforcement learning optimizes pitch angle for lower energy consumption • Observer-based control ensures stable flight under varying disturbances • Simulations show major energy savings and smoother trajectory tracking The problem of energy-efficient control for tilt-rotor quadrotor unmanned aerial vehicles (UAVs) operating under dynamic wind disturbances remains relatively underexplored. To address this gap, we develop a novel control framework that builds upon a simplified two-dimensional (2D) dynamic model of the tilt-rotor quadrotor, in which computational fluid dynamics (CFD) analysis is employed to determine the aerodynamic coefficients as functions of the angle of attack. Based on this model, the proposed framework consists of three parts: 1) a disturbance observer-based backstepping (DO-BS) controller to ensure robust trajectory tracking under wind disturbances, 2) a reinforcement learning (RL) agent employing the twin delayed deep deterministic policy gradient (TD3) algorithm to determine the optimal pitch angle in real time and minimize energy consumption, and 3) a Butterworth filter to produce smooth derivatives of the pitch-angle reference while preventing the explosion of complexity caused by derivative computations. Numerical simulations conducted under three representative cases demonstrate that the proposed approach achieves approximately 12–18% reduction in energy consumption compared with the extremum seeking control (ESC) method, and 33–49% relative to a conventional quadrotor. Furthermore, tracking performance is improved across all cases, with moderate reductions under nominal conditions and substantially larger improvements under realistic scenarios that include sensor noise, actuator dynamics, and atmospheric wind disturbances modeled using the Dryden wind model, indicating enhanced trajectory tracking accuracy and smoother system responses. In summary, the proposed framework provides faster convergence and improved adaptability for energy-efficient pitch-angle optimization, effectively improving flight stability and endurance.