Investigating the effects of recursion in convolutional layers using analytical methods

Most Convolutional Neural Networks (CNNs) consist of a number of stages of decreasing spatial resolution and increasing channel dimension between succeeding stages, each stage is composed of convolutional blocks that are repeated a number of times. Previous research on very simple CNNs consisting of a number of convolutional layers in each stage demonstrated that the introduction of feedback loops around convolutional layers can improve results. This paper studies the effectiveness of recursion on convolutional blocks in a more general setting and aims at explaining the results. Four recent models, namely ResNet, Inception, MobileNet and DenseNet are considered in this study. It is found that for all but DenseNet, the recursive version produces results that are similar or better than their feedforward counterpart when the number of convolutional blocks are preserved. To understand this finding and to discriminate the functional behaviors of the feedforward and recursive counterparts, we embark on three investigations: (1) measuring the evolution of the contextualization of the neurons of the last layer using the effective receptive field concept; (2) comparing the position and the size of the global coverage of the networks using class activation maps; and (3) analyzing the evolution of the organization of the feature space prior to the classifier using the Silhouette score. The investigations reveal that the recursion of a convolutional block shares many similarities with the behavior of a sequence of that block, indicating that a recursive alternative consisting of a single physical layer, can be regarded as a “faithful simulation” of its deeper‘feedforward counterpart. We conclude that except for densely connected models, the recursion of convolutional blocks is a safe and powerful alternative enhancing modern network architectures.