HyDiT : Heterogeneous Multi-Teacher Knowledge Distillation for Compressing Vision Transformers in Autonomous Electric Vehicles

Autonomous Electric Vehicles (AEVs) require traffic sign classification models that can operate under limited computational and memory resources. This paper presents HyDiT, an offline, response-based, heterogeneous multi-teacher knowledge distillation framework for training a compact Vision Transformer (ViT) student. The teacher pool combines CNN-based and Transformer-based models so that the student receives supervision from different model families. Two teacher configurations are evaluated: ResNet50 with Swin-Transformer V2, and ResNet152 with a pre-trained ViT. During training, batchwise loss-based weights are used to combine the teacher prediction outputs before transferring them to the distillation head of the student. The term "heterogeneous" refers only to the composition of the teacher pool; the student architecture remains a compact ViT and does not combine CNN and Transformer layers. On the GTSRB dataset, the resulting student contains 5.97 million parameters and requires approximately 1.35 G MACs (about 2.70 GFLOPs). The performance evaluation shows that, under the common GT-SRB pipeline used in this study, the heterogeneous multi-teacher student achieves the highest test accuracy among the evaluated baseline ViT and single-teacher student configurations; its test accuracy is 90.34%. These results examine whether supervision from heterogeneous teacher models can improve a compact ViT student while retaining a substantially smaller inference architecture.