The Null Layer: increasing convolutional neural network efficiency

Convolutional neural networks are currently used in many applications; however, their construction involves a sequence of choices that can drastically affect the final network accuracy. In addition to the choice of architecture and hyperparameters, weight initialization of the layers is an essential step. We analyzed two different initialization methods in the layers to study their impact on network accuracy. The proposed method, the Null Layer, has a weight initialization of the first convolutional layer equal to zero, whereas the other layers have another type of initialization. The second method, the traditional method, uses the same weight initialization for all layers. Three different networks, four datasets, five activation functions, and three weight initializations were used for the tests. The results showed that the Null Layer method is an efficient approach for increasing network accuracy. It presented better accuracy in 53% of the tests than the traditional method without additional computational cost.