Symbolic representation of objects relative poses for robotic manipulation tasks

Collaborative robots (cobots) are democratizing industrial automation with their user-friendly programming approaches. Nevertheless, the Blockly-like interfaces typically available on cobots still require the user to define the program logic flow. Recent advancements in robotics research provide the robotic system with the reasoning capabilities given by symbolic artificial intelligence. This way, the cobot can acquire a new skill from a user demonstration, understand its semantics, and use symbolic planning for grounding and sequencing. Such methodologies rely on a symbolic description of the scene that should adequately represent how the cobot’s actions modify the environment. The symbols employed in the literature, however, either lack descriptive accuracy or are too specific for the targeted task, resulting in the application of the proposed teaching methodologies only to simple scenarios. This paper addresses these issues by introducing a methodology for symbolically describing general-purpose spatial relations between entities in a workspace, enhancing the flexibility and the range of application of cobots symbolic reasoning for complex manipulation tasks. The proposed approach involves defining a tunable set of predicates for relative positions and orientations, enabling precise symbolic representations, necessary for real-world tasks. The adoption of these symbols into a Programming by Demonstration framework empowers non-expert users to teach skills and deploy cobots in complex industrial tasks without coding. Experimental results demonstrate the effectiveness of this method, showing that first-time users can deploy cobots for a complex machine tending task comprising parts reorientations. • Application of semantic-informed Programming by Demonstration for industrial tasks. • Definition of tunable, general-use quantitative symbols for relative positions. • Development of a strategy for representing relative orientations symbolically. • Validation through a user study on a machine-tending task with part reorientations.