Vision-Guided Robotic Arm with Deep Learning and Secure Digital Twin for Pre-Deployment Quadruped Robot Operation

In this article, we propose a vision-guided robotic arm system that integrates deep learning, digital twin, and cybersecurity to enhance the maintenance of quadruped robots. The proposed system comprises two core components. (1) A YOLOv11-based object detection system with depth cameras identifies maintenance targets and extracts 3D coordinates for autonomous robotic arm localization and manipulation. (2) A digital twin environment is used to simulate and validate path planning, trajectory stability, and operational safety before real-time deployment. Moreover, we propose a secure communication system to enhance system reliability and resilience against cyber–physical threats. To verify system performance, we first used a mug as a test object to validate the robotic arm’s posture control, motion accuracy, and trajectory consistency. The platform was then extended to quadruped robot maintenance. Experimental results show that the proposed YOLO11 model achieved an [email protected] of 0.995 with reduced computation time. In addition, real experiments demonstrated that the system accurately located the target, with average planar and depth deviations of 0.0277 m and 0.0301 m, respectively. Following the physical proxy validation, the workflow was extended to a simulated quadruped-robot maintenance scenario, in which screw access reachability, trajectory feasibility, and collision-free manipulator motion were evaluated. DIATwin was used for kinematic modeling, trajectory generation, and physical robot script generation, whereas NVIDIA Omniverse was employed as a high fidelity visualization and physics-verification environment using the URDF model exported from DIATwin. The trajectory-transfer experiments demonstrated that digital-twin-generated paths could be successfully transferred to and executed by the physical robotic arm across the evaluated test configurations. Therefore, the proposed system demonstrates the feasibility of integrating vision-guided manipulation, secure communication, and digital-twin-based pre-deployment verification for future smart, automated, and cyber–physical robotics systems.