Optimising Neural Networks with Fine-Grained Forward-Forward Algorithm: A Novel Backpropagation-Free Training Algorithm

This paper introduces the Fine-Grained Forward-Forward Algorithm (FGFF), a novel backpropagation-free network training methodology that builds upon the foundational Forward-Forward algorithm [1]. FGFF takes full advantage of being entirely backpropagation-free and enhances neural network performance through the implementation of a fine-grained significance matrix. This matrix quantitatively assesses the contribution of each neuron’s activity towards the overall goodness function, thereby facilitating more explorative and precise weight adjustments. The efficacy of the algorithm is demonstrated by comparing its accuracy with the vanilla Forward-Forward algorithm and the standard backpropagation methods. Our evaluations employ three benchmark datasets: MNIST, FashionMNIST, and CIFAR-10 [2], [3], [4]. The results indicate that the FGFF achieves superior performance over the Forward-Forward algorithm, specifically showing a testing accuracy improvement of up to 25.54% with no obvious cost in the computation time, underscoring its enhanced capability in handling complex pattern recognition tasks.