Optimizing Real-Time Image Classification Models on Jetson Nano Using Self-Supervised Pre-Training

Under the background of high evolving of image processing technology characteristics and deep learning skills, how to implement both high-precision and real-time image classification tasks based on resource-constrained Jetson Nano platform has become a hot research issue. We then leverage a combination of lightweight convolutional network and Transformer as our backbone model, performing self-supervised pre-training in a label-free way via contrastive learning on large-scale data in order to acquire generalizable visual representations. Then the pre-trained general representation are efficiently coupled with the classification branch of the model with the multimodal feature fusion as well as the multi-task decoupling. In this procedure, we additionally propose knowledge distillation and attention-based feature weighting mechanisms to reduce the inference overhead and guarantee the expressiveness of the model. Finally, we realized a deep model by sequentially applying gradient clipping, packet convolution optimization, and dynamic quantization techniques on Jetson Nano, preserving complex network structure with the requirement of low power consumption, achieving the effective balance of high accuracy and real-time performance. The experimental results demonstrate that the proposed model can obtain a notable gain of accuracy on similar models while maintaining the real-time inference speed on Jetson Nano.