Exploring reward shaping in discrete and continuous action spaces

In robotics, reinforcement learning can train controllers or agents to find optimal solutions for complex tasks by enabling the robot to interact repeatedly with the environment. The reward function is an important aspect that guides any reinforcement algorithm to find the desired solution successfully. This work examines two deep reinforcement learning approaches, one uses a Deep Q-Network(DQN) and the other a deep deterministic policy gradient (DDPG) algorithm, applicable to navigation tasks for mobile robots. Comparison between different reward schemes for both algorithms is one of the main focuses in this work. The methodology is implemented in the simulation for a mobile robot called TurtleBot3. The task for the robot is to navigate through obstacles from an initial location to a goal position. Finally, the trained end-to-end navigation stack is also implemented on the actual TurtleBot3 in a real environment. The robot uses a Lidar sensor to detect obstacles. The Lidar measurements and the relative position and angle of the robot to the target location are the inputs to the controller. The TurtleBot3 also utilizes distance information from its Lidar sensor to create an environmental map using the simultaneous localization and mapping (SLAM) technique. Additionally, given an initial position, the robot employs its inertial measurement unit (IMU) sensor and encoders for precise localization.