Humanoid Robotics Powered by Vision-Based LLM Zero Shot Interactions

Advancements in artificial intelligence (AI) have transformed robotics by enabling systems to autonomously execute complex tasks with minimal human involvement. Traditional methods, however, often depend on costly hardware, continuous monitoring, and intricate software integration, which constrain scalability and widespread implementation. This research presents an innovative approach that combines vision-based large language models (LLMs) with zero-shot prompting to autonomously program robotic systems, including a humanoid robot equipped with dual manipulators. The proposed system harnesses contextual image data to efficiently generate task-specific code, eliminating the need for iterative corrections. Training is conducted through OpenAI's Assistant feature, utilizing documents predominantly comprising images, while continuous operation is facilitated by a self-looping mechanism. Experimental results highlight the system's capability to perform manipulator tasks with notable accuracy, paving the way for scalable, adaptive, and dynamic automation. This study addresses both practical and theoretical challenges in automation, providing a cost-effective framework for next-generation robotic systems.