Integration of Functionless Large Language Model and Memristor-Inspired Visual Servoing Algorithm for Object Approach and Contact

In computer vision-based robotic object grasping tasks, the use of natural language as inputs to the large language model to generate motion commands for robotic systems often leads to errors in function matching. Moreover, even for the case that the large language model successfully generates suitable motion commands, the object grasping task may still experience a failure due to movement of the object by the user. To cope with the aforementioned difficulties, this paper proposes a VS_LLM scheme that consists of a memristor-inspired visual servoing scheme and a Functionless LLM (FLLLM). Experimental results show that the proposed FLLLM reduces the number of output tokens generated by the model. Physical robot experiments demonstrate the end-effector contact with stationary and moving targets, as well as improved smoothness of commanded acceleration under specific conditions.