Accelerated Neural Network Initialization via Training of Symmetrically Partitioned Models

Abstract This paper proposes a novel approach to initializing neural network weights. The algorithm relies on cutting the inputs by changing the original shapes into symmetrical slices and cutting the models vertically. These are trained in parallel with the potential for distribution between computing nodes. The resulting weights are concatenated in a larger model, increasing accuracy metrics for the training epochs. Further training on it improves metrics for the first epochs while contributing to the final accuracy. This approach can reduce the computational demands of training a model on a single device. Additionally, the time required to train the concatenated model is reduced.