In-situ forward sparse training of large-scale optical neural networks

The rapidly growing scale of neural network models requires more energy-efficient computing hardware to meet computational demands. Optical neural networks (ONNs) are particularly appealing owing to their potential for high parallelism, fast dynamics, and low energy consumption. However, training large-scale ONNs efficiently remains challenging due to the heavy reliance on conventional electronic platforms for modeling and optimization, which incurs substantial computational costs and energy overheads, thereby undermining the inherent advantages of optical computing. Here, we present an in-situ forward sparse training (IFST) framework that can optimize large-scale ONNs by performing the majority of computations optically. Our approach leverages optical forward inferences to perform parallel gradient calculations without dependence on digital models, thus significantly increasing the proportion of optical operations during training. IFST combines sparsity with optical parallelism using parameter activation masks designed by physical priors and dynamic gradient pruning to reduce the dimensionality of the optimization space and computational complexity, while maintaining or even improving performance, making it particularly suitable for scaling ONNs. We validate the effectiveness of IFST by applying it to diffractive ONNs in image classification and segmentation tasks, achieving performance comparable to the ideal backpropagation method while significantly enhancing energy efficiency. Furthermore, we demonstrate that ONNs trained with IFST maintain robust performance in dynamic environments.