Research on Hyperparameter Optimization Strategy of Lightweight ViT Model Based on SVM

This paper investigates a lightweight Vision Transformer (Vi <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex>) model based on Support Vector Machine (SVM) optimization, using methods such as structural optimization and hyperparameter optimization that introduce SVM into the ViT model, with the aim of improving the model's performance in target detection tasks. This study addresses several aspects of the lightweight ViT model structure design, SVM optimization strategy, and analysis of the total optimization process. First, SVM is introduced into the structural design of the Vi <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex> model to optimize the feature extraction process. Second, by optimizing the hyperparameters of the ViT model, the training process of the model is accelerated with the powerful classification ability of SVM. Finally, the overall optimization process is analyzed and experimentally validated to demonstrate the superior performance of the optimized model in the target detection task on standard datasets such as the COCO dataset. The introduction of Support Vector Machines (SVMs) into the Vision Transformer (ViT) model allows the model parameters to be effectively reduced in resource-constrained scenarios, improves the training and inference speeds, and significantly improves the overall performance of the model.