Efficient Task Organization with Commonsense Knowledge for Human-Robot Collaborative Tasks

We present a new and innovative approach called DISCERN (Detection Image System with Commonsense Efficient Ranking Network) to “discern” object selection priority designed for human-robot collaborative tasks. Our approach utilizes a combination of standard image models, a commonsense knowledge base (CSKB), a vision language model, and custom priorities derived from human intuition to determine an optimal order for the robot’s actions. DISCERN is a competitive solution to extensive training or learning from human demonstrations and works out-of-the-box with effective results and minimal resources, hence implying low algorithmic complexity and high execution efficiency. We validated the proposed approach in a typical human-robot collaborative home dining table cleaning task, although they can be applied to any household setting. Experimental results and evaluations demonstrate that the developed DISCERN has significantly better performance than baseline methods.