Real-Time Explainable Vision-in-the-Loop Control of a Planar 2RRR Parallel Manipulator Using a Digital-Twin Reference

This work presents and experimentally characterizes a real-time, explainable Eye-to-Hand visual-feedback architecture for trajectory control of a planar 2RRR parallel manipulator. A fine-tuned YOLOv8n detector locates an end-effector marker, calibrated homography converts image coordinates into Cartesian position, and the resulting visual error corrects the commanded trajectory before inverse kinematics generates the actuator commands. A previously validated ROS/Gazebo digital twin (DT) provides the kinematic model used for reference generation and validation, while the Dynamixel MX-64 encoders are retained only as an independent measurement source for training, calibration, and experimental validation. On 3142 test images, the observer achieved a position RMSE of 3.455 mm (MAE = 2.804 mm). Wilcoxon tests detected no significant median offset on either Cartesian axis, although confidence intervals for the arithmetic mean differences excluded zero and indicate small mean offsets. The closed loop was then characterized over 24 physical runs and four trajectory geometries in two configurations: continuous execution with independent visual monitoring (pooled RMSE 1.78 mm, n=4608) and segment-wise visual correction (pooled RMSE 3.91 mm, n=4716). A dedicated ablation isolated the contribution of the correction itself under identical segmentation: 4.257 mm with correction against 4.331 mm without it. The visual correction therefore improves tracking by 0.074 mm, whereas the trajectory interruption required to apply it costs 2.13 mm. Grad-CAM, LIME, and SHAP concentrated their attribution on the end-effector marker during successful detections and lost spatial agreement under synthetic occlusion. The contributions of this work are a vision-based state observer validated against an independent encoder channel, a multi-method explainability audit of that observer, and a quantified assessment of when segment-wise visual correction is worth its cost on a lightweight parallel mechanism whose tracking accuracy depends on uninterrupted motion. Physical validation was performed on the Dynamixel-equipped prototype; operation with actuators lacking joint-state telemetry remains to be investigated.