Photonic neuromorphic processor with high energy efficiency exceeding 100 GOPS/W/mm2

Photonic neuromorphic computing offers substantial enhancements in machine vision processing by providing ultrahigh operation bandwidth and reduced energy consumption, thereby outperforming conventional electronic systems based on von Neumann architectures. However, scalability challenges persist in implementing chip-scale photonic computing—particularly when accommodating high-dimensional tensor inputs—due to inherent physical constraints and the complexity of control engineering. In this paper, we introduce a compact photonic neuromorphic processor that integrates with an on-chip diffractive multi-channel multi-kernel optical convolution unit (M2OCU) to enable parallel, high-complexity vision perception. By leveraging amplitude-phase co-modulation within the M2OCU, high-dimensional tensors can be simultaneously loaded and processed with energy efficiency exceeding 100 giga-operations per watt per square millimeter (GOPS/W/mm2). We experimentally validate the feasibility of M2OCU by demonstrating its application in machine vision, including photonic channel-wise pooling and video-based multi-frame fusion for human action recognition. Both applications achieve comparable accuracies while significantly reducing computational load. Our work provides a pathway for future large-scale and high-dimensional information processing with photonic integrated circuits, enabling the practical application of chip-scale photonic computing in increasingly sophisticated scenarios—such as autonomous driving, astronautics, and telecommunications—at picosecond latency and femtojoule-level energy consumption per operation.