Evaluating Convolutional Neural Networks via measuring similarity of their first-layer filters and Gabor filters

Convolutional Neural Networks (CNNs) have reached a certain level of technological maturity and researchers are now focusing on what they learn, how they work, why, and how to further improve their performance. For example, incorporating Gabor filters - a mathematical representation of the responses in the mammalian primary visual corte-into CNNs has been shown to boost CNN performance in terms of accuracy. Motivated by the ability of Gabor filters to increase the performance of CNNs and the importance of proposing diverse analytical tools to facilitate CNN evaluation and modeling, we proposed a novel three-step methodology specially crafted to systematically analyze and evaluate vision CNNs in terms of the similarity of their first-layer filters to Gabor filters. It is based on direct similarity measurements, and also indirect measurements of similarity through the impact of filter replacement on performance (accuracy) and security (resilience to adversarial attacks). The methodology demonstrates great potential for the areas of explainability, evaluation and modeling of CNN filter responses. Our results showed that all first-layer filters formed in the examined CNNs are highly similar or almost identical to Gabor filters, which was confirmed by our direct and indirect measurements. The security-related experiment further showed that although the filter replacement does not fully protect the network against the perturbations from the baseline model, it can slightly improve its resilience suggesting potential practical implications for security. We provide our code at https://github.com/iitis/Gabor-Replacement.