Deep Reinforcement Learning Based Parameter Optimization for Processing Curved Surfaces

The planning of robotic surface processing tasks, such as polishing, cleaning, and painting on 3D objects with varying shapes and sizes, remain challenging. Traditional heuristic methods often face computational limits, while supervised approaches such as learning from demonstrations require significant amounts of expert data for each task. Although reinforcement learning has shown success in handling fat or subtly curved surfaces, planning of process on surfaces with complex and diverse shape remains challenging due to their geometric variability, the high dimensionality of the action space, and the need to adapt tool angles to varying surface curvatures. In our work, we show how two process parameters, relevant for a polishing process, can be derived from the geometry of the workpiece and the tool in a time-efficient manner. Given a process trajectory defined by intermediate surface path points, we learned a deep reinforcement learning-based policy to estimate parameters related to the inclination and orientation of a polishing tool such that collisions of workpiece and tool are avoided, the polished surface is maximized, and the tool angles are as stable as possible along the path. We further evaluated our method on various unseen and diverse basin geometries in simulation and also compared it with an iterative sampling-based method.