Modeling and Analysis of the 8 Filters from the “Master Key Filters Hypothesis” for Depthwise-Separable Deep Networks in Relation to Idealized Receptive Fields Based on Scale-Space Theory

Abstract This paper presents the results of analyzing and modeling a set of 8 “master key filters”, which have been extracted by applying a clustering approach to the receptive fields learned in depthwise-separable deep networks based on the ConvNeXt architecture. For this purpose, we first compute spatial spread measures in terms of weighted mean values and weighted variances of the absolute values of the learned filters, which support the working hypotheses that: (i) the learned filters can be modeled by separable filtering operations over the spatial domain, and that (ii) the spatial offsets of the those learned filters that are non-centered are rather close to half a grid unit. Then, we model the clustered “master key filters” in terms of difference operators applied to a spatial smoothing operation in terms of the discrete analog of the Gaussian kernel, and demonstrate that the resulting idealized models of the receptive fields show good qualitative similarity to the learned filters. This modeling is performed in two different ways: (i) using possibly different values of the scale parameters in the coordinate directions for each filter, and (ii) using the same value of the scale parameter in both coordinate directions. Then, we perform the actual model fitting by either (i) requiring spatial spread measures in terms of spatial variances of the absolute values of the receptive fields to be equal, or (ii) minimizing the discrete $$l_1$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>l</mml:mi> <mml:mn>1</mml:mn> </mml:msub> </mml:math> - or $$l_2$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>l</mml:mi> <mml:mn>2</mml:mn> </mml:msub> </mml:math> -norms between the idealized receptive field models and the learned filters. Complementary experimental results then demonstrate that the idealized models of receptive fields have very good predictive properties for replacing the learned filters by idealized filters in depthwise-separable deep networks, thus showing that the learned filters in depthwise-separable deep networks can be well approximated by discrete scale-space filters. Notably, we show that, for a reduced version of the ConvNeXt architecture, using a set of only 8 discrete scale-space filters leads to almost as good accuracy as for the receptive fields trained from scratch on the ImageNet dataset.