Temporal Clustering and Temporal Reference Based Specular Detection For 1-MS Visual Feedback System

The 1-ms visual feedback system is critical for seamless actuation in robotics, as any delay affects its performance in handling dynamic situations. Specular reflections cause problems in many visual technologies, making specular detection crucial in 1-ms visual feedback systems. However, existing real-time methods, which target Neumann architecture, fail to achieve the 1-ms delay due to spatial memory paths resulting from extensive frame-based processing. This research aims to develop a 1-ms specular detection system from both algorithm and architecture perspectives, proposing 1) temporal clustering and temporal reference based specular detection method, which leverages temporal domain information to address the requirements of frame-based processing; and 2) global-local integrated specular detection architecture, which enables the coexistence of local and global processing within a 1-ms stream-based architecture. The proposed methods are implemented on FPGA. The evaluation shows that the proposed system supports sensing and processing a $1000-\mathrm{fps}$ sequence with a delay of $0.941 \mathrm{~ms} /$ frame.