Efficacy of Two Hidden Layers Artificial Neural Network Synapticity for Deep Learning: A Case of Pattern Recognition

Most research works in Artificial Neural Network (ANN) are accustomed with the use of single hidden layer (SHL) topology without giving considerations to the problem type, its complexity and desired depth of supervised or unsupervised learning. This could be partly due to the inherent complexities associated with the use of more than one hidden layer which in turn affects solution efficiency. However, the trade-off occasionally is between efficiency and effectiveness of result. When effectiveness is prioritized perhaps for sensitive or mission critical systems, then multiple hidden layers can become advantageous. This research has investigated the ability of an Artificial Neural Network (ANN) with two hidden layer topology to exhibit deep learning behaviour in comparison with a single hidden layer architecture ANN system. A two hidden layer (THL) Neural Network was developed and implemented using Microsoft Visual Studio programming suite and applied to a pattern recognition problem. The gradient descent optimization of the back propagation algorithm in a feed forward scheme was used in the development of the supervised ANN which consisted of thirty inputs at the input layer, two hidden layers with five nodes and a single output layer with one node for a Boolean response. Normalized images mapped into a pattern extraction template using principal component analysis (PCA) of the original images served as pre-processed inputs to the two hidden layer architecture with an initial learning rate of η = 0.1 and maximum tolerable rate of η = 0.4 for fast convergence. Iterations for validation of the feed forward back propagation algorithm using three image patterns showed that over 96% recognition of presented data was recorded. Graphical comparison of the results obtained from separate iterative sessions of the One Hidden Layer (OHL) and (THL) architectures under same input-output dataset revealed more visible traits of attained deep learning by the two hidden layer architecture due to enhanced synapticity of additional nodes.