A gradient-free training approach for optical neural networks based on stochastic functions

Due to increasingly large computational resources, modern neural networks are severely constrained due to their processing speed and energy consumption. Optical neural networks (ONNs), which use photonic structures to process signals at the physical level as an alternative to the computation in the electronic domain provided by traditional neural networks, are an attractive approach to implementing ultra-high-speed, low-energy parallel computation. Nevertheless, current training processes for electronic domain neural networks are optimized from gradient-based training methods, such as backpropagation, not compatible with ONNs with gradient-free features. In this work, a stochastic function-based gradient-free training method, i.e., stochastic function direct feedback alignment (SF-DFA) is demonstrated and evaluated. SF-DFA trains a gradient-free system using stochastic matrices and functions to replace the weights and gradients of the nodes in neural networks. Thus, it is feasible to train ONNs without a prior knowledge of the photonic system and its gradients. In addition, implementing such training process on optical hardware is also known to be possible. A series of studies have been carried out for a spectral slicing neural network (SS-NN) architecture trained by SF-DFA. The SS-NN system uses bandpass filters embedded in optical fiber micro rings to enable slicing of the optical signal spectrum. Our results demonstrate that the training of ONN using SF-DFA can converge efficiently, with higher processing speed and lower energy consumption compared to back-propagation.