PPO-Based Reinforcement Learning Control of a Flapping-Wing Robot with a Bio-Inspired Sensing and Actuation Feather Unit

Bio-inspired flow-sensing and actuation mechanisms offer a promising path for enhancing the stability of flapping-wing flying robots (FWFRs) operating in dynamic and noisy environments. This study introduces a bio-inspired sensing and actuation feather unit (SAFU) that mimics the covert feathers of falcons and serves simultaneously as a distributed flow sensor and an adaptive actuation element. Each electromechanical feather (EF) passively detects airflow disturbances through deflection and actively modulates its flaps through an embedded actuator, enabling real-time aerodynamic adaptation. A reduced-order bond-graph model capturing the coupled aero-electromechanical dynamics of the FWFR wing and SAFU is developed to provide a physics-based training environment for a proximal policy optimization (PPO) based reinforcement learning controller. Through closed-loop interaction with this environment, the PPO policy autonomously learns control actions that regulate feather displacement, reduce airflow-induced loads, and improve dynamic stability without predefined control laws. Simulation results show that the PPO-driven SAFU achieves fast, well-damped responses with rise times below 0.5 s, settling times under 1.4 s, near-zero steady-state error across varying gust conditions and up to 50% alleviation of airflow-induced disturbance effects. Overall, this work highlights the potential of bio-inspired sensing-actuation architectures, combined with reinforcement learning, to serve as a promising solution for future flapping-wing drone designs, enabling enhanced resilience, autonomous flow adaptation, and intelligent aerodynamic control during operations in gusts.