Vision on the Warehouse Floor: Perceptive HRI for Autonomous Forklifts

Autonomous forklifts in dynamic industrial facilities must simultaneously solve precise pallet perception and safe, accountable human–robot interaction, challenges that have rarely been tackled together on real edge hardware. To address this, we propose an RGB-only six-degree-of-freedom (6D) pallet pose estimation pipeline optimized for real-time deployment on an onboard edge device and trained entirely on a synthetic dataset. On three real industrial test sequences our method achieves an average area under the accuracy–threshold curve (AUC) of 0.484, compared with 0.085 for the Deep Object Pose Estimation (DOPE) baseline. Hence, we drastically reduce the sim-to-real gap that had until now prevented the use of purely synthetic training in this domain. The system is completed by two human–robot interaction (HRI) modules: a skeleton-based gesture recognizer and a face recognition pipeline with role-based access control including a safety-by-design protocol that keeps the emergency stop accessible to any person regardless of authorization state. Full evaluation on the vehicle in a real-world intralogistics facility demonstrates real-time end-to-end operation across all three modules, from gesture-based login and operator identification to autonomous pallet engagement.