Wave-Bottleneck: an Enhanced Architecture for Decreasing Parameters in Deep Network Models

The field of computer vision has seen remarkable advancements with deep neural networks, particularly the ResNet architecture, which effectively reduces the number of parameters and addresses the vanishing gradient problem through its Bottleneck block design and skip connections. In this study, we strive to further reduce network parameters while safeguarding model precision by introducing the Wave Bottleneck. This groundbreaking integration seamlessly merges the Bottleneck block with Discrete Wavelet Transform (DWT). Wave Bottleneck leverages DWT for feature decomposition and reconstruction, combining 1D and 2D DWT to enhance computational efficiency. Our method performs feature extraction on the input data in the channel domain to efficiently and effectively capture the correlation and important features between channels. Moreover, we realize important feature extraction in the spatial domain by performing feature extraction. Extensive experiments on standard datasets show that Wave Bottleneck outperforms traditional ResNet models in terms of accuracy and parameter reduction, showcasing its potential for deployment in resource-constrained environments.