Cross-domain knowledge distillation for integrated optical neural networks: toward robust and compact learning with pseudo-zero-shot capability

This paper proposes what we believe to be a novel training framework for on-chip optical neural networks (ONNs), termed cross-domain electro-optical knowledge distillation (CDEOKD). Unlike conventional training methods that rely solely on original labeled data, CDEOKD employs a high-performance digital neural network as a teacher model to generate informative soft labels, providing enhanced supervision for the student-ONN. We systematically evaluate this approach on fully connected Mach-Zehnder interferometer (MZI)-based ONNs across various scales. The results demonstrate that for scale-limited ONNs, the recognition accuracy on the Fashion-MNIST dataset is consistently and significantly improved. In particular, for a 10 × 10 × 2 ONN (i.e., with 10 input/output ports and 2 cascaded layers based on the Clements architecture), our method achieves a 27.36% increase in accuracy compared to traditional categorical cross-entropy training. Furthermore, the proposed framework exhibits strong robustness against phase deviations and MZI losses, and maintains stable performance even under low-precision (4-bit) weight quantization, highlighting its suitability for practical photonic hardware deployment. Additionally, the framework enables pseudo-zero-shot learning capabilities in integrated ONNs, further expanding their potential for real-world applications.