Improved Classification with Data Augmentation for Attention Based Residual Neural Networks

Neural networks work best when there is a large dataset. But large dataset with large number of classes introduces decrease in accuracy in prediction for shallow neural networks. To overcome this issue, large hidden layers are required to be added in a neural network. Thus deep neural network is suited for handling large number of classes. Again, adding more layers comes with a cost of high number of computations, also it brings in with itself new issues of vanishing and exploding gradients. Problem of vanishing and exploding gradients can be solved by introducing new identity connections in a neural network. Such neural networks are known as residual neural network or in short Res-Net's. Res-net's have proved to handle vanishing or exploding gradients. Image augmentation of dataset has shown to improve accuracy in comparison with simple raw data. Similarly attention mechanism has its own accuracy benefits in neural networks. We explore primary method of mix-up augmentation with some standard methods of image augmentation and analyze accuracy benefits if any. Attention blocks are added to the network to achieve further accuracy benefits. A metric of losses and accuracy is built with iterations, which are commonly known as epochs, for assisting us to comprehend the performance.