Lightweight Fuss-Free Network-Based Crowd Counting Model Using Knowledge Distillation

This paper presents FFNet-S, a lightweight crowd counting model built on the simple and efficient architecture of FFNet, but enhanced via knowledge distillation (KD). The student model employs MobileNetV3 as the backbone with preservation of the multi-scale feature fusion structure of FFNet. To guide the student effectively, a composite distillation loss is introduced. This combines soft target regression, intermediate feature alignment, and attention transfer. A two-stage training strategy is adopted. Initial training on the ground truth ensures stable convergence. Next, gradual incorporation of distillation losses enhance performance. Experiments on benchmark datasets, including the ShanghaiTech Part A (SHA) and Part B (SHB), show that FFNet-S is over 90% smaller than the teacher model, but the accuracy is comparable. Moreover, FFNet-S makes inferences in real time, rendering it suitable for deployment on edge devices with limited computational resources. The proposed approach shows that a carefully designed KD framework enables compact models to exhibit the capacities of larger more complex networks without a significant loss of accuracy. Balancing of speed, accuracy, and efficiency renders FFNet-S very applicable in real-world scenarios such as surveillance systems, drones, and Internet of Things platforms. We present a practical and scalable solution for efficient crowd counting. This encourages further exploration of lightweight models for computer vision tasks when resources are constrained.