Augmented Perception: a real-time digital twin based approach to enhance robotic perception

This paper introduces an Augmented Perception (AP) framework to enhance robotic perception in resilient manufacturing systems (MS) by integrating Digital Twin (DT) data directly at the sensor level in real time. Inspired by augmented reality, this approach enables robots to perceive both physical and virtual entities within a unified representation. To ensure real-time performance, we address the critical challenges of low-latency data processing and seamless integration. To evaluate the system’s real-time compliance, we propose a latency test and validate our approach through simulations using the Tiago++ robot in Gazebo. Results demonstrate that AP maintains efficient sensor-level integration while meeting real-time processing requirements.