An Analysis of the Reward Function of a Hexapod Robot

Research into robotics makes extensive use of hexapod robots because they display excellent stability while navigating. This paper offers an optimized reward function for reinforcement learning to improve gait performance of these robots with regard to stability, energy efficiency, and accurate motion control. The research uses Proximal Policy Optimization (PPO) that covers the Brax simulation environment to train a hexapod model based on a reward function including six elements. Forward motion reward, healthy posture maintenance, lateral/vertical velocity penalties, torque control, and orientation stability. Six different gait cases indicate that stable and energy-efficient walking is guaranteed when the balanced reward weights are used, while the relaxed parameters improve adaptability. Some of the important outcomes of the work are a speed of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$4 ~\mathrm{m} / \mathrm{s}$</tex> or less in an optimized configuration and less actuator torque in an optimized configuration. The framework allows for autonomous learning of adaptive locomotion without predefined gait patterns. Quantitative analysis reveals that stability constraints highly enhance directional control, and unnecessary movements are not done. This work adds a validated design in reward functions and comparative information on gait dynamics in addition to efficient hexapod robots for deployment in search-and-rescue and automated transportation.