Real-time all-directional 3D recognition and multidistortion correction via prior diffraction neural networks

Robust three-dimensional (3D) recognition across different viewing angles is crucial for dynamic applications such as autonomous navigation and augmented reality; however, the application of the technology remains challenging owing to factors such as orientation, deformation, and noise. Wave-based analogous computing, particularly diffraction neural networks (DNNs), constitutes a scan-free, energy-efficient means of mitigating these issues with strong resilience to environmental disturbances. Herein, we present a real-time all-directional 3D object recognition and distortion correction system using a deep knowledge prior DNN. Our approach effectively addressed complex two-dimensional (2D) and 3D distortions by optimizing the metasurface parameters with minimal training data and refining them using DNNs. Experimental results demonstrate that the system can effectively rectify distortions and recognize objects in real time, even under varying perspectives and multiple complex distortions. In 3D recognition, the prior DNN reliably identifies both dynamic and static objects, maintaining stable performance despite arbitrary orientation changes, highlighting its adaptability to complex and dynamic environments. Our system can function either as a preprocessing tool for imaging platforms or as a stand-alone solution, facilitating 3D recognition tasks such as motion sensing and facial recognition. It offers a scalable solution for high-speed recognition tasks in dynamic and resource-constrained applications.