Optimizing Kernel Configurations and Convolutional Strategies for Efficient Shallow CNNs in Real-Time Vision Systems

This study investigates the impact of kernel size configurations and convolutional layer types on the performance of shallow convolutional neural networks (CNNs) for image classification tasks. This study aims to iterate over some kernel variations and to propose alternatives to well-known configurations. We concentrate on investigation of the effect of applying different kernels by performing analysis on fixed-size shallow network. The shallowness provides a background to assess kernel modification impact by minimizing the number of trainable layers. Using the MNIST and CIFAR-10 datasets, we systematically analyze the effects of kernel sizes (ranging from 3x3 to 11x11 in both layers) and convolutional configurations, including standard, dilated, and strided convolutions, across multiple input resolutions (32x32, 64x64, 128x128 and 256x256 pixels). The results reveal that for simple datasets like MNIST, kernel size has minimal impact, with most configurations achieving high accuracy (~97-98%). In contrast, for CIFAR-10, smaller kernels in the first layer are more effective for capturing fine-grained details, while larger kernels in the second layer improve generalization at higher resolutions. Dilated convolutions demonstrate significant advantages when dealing with high-resolution images by reducing inference time (almost 2x speedup for 256x256 images) and preserving accuracy, especially for images with high spatial redundancy. Strided convolutions, while computationally efficient, exhibit consistent accuracy degradation due to loss of spatial detail. Future research directions may include hybrid designs leveraging dynamic kernel configurations to further enhance performance across diverse datasets and resolutions.