Low-Cost FPGA-Based Hybrid Defence Against Sticker Attacks for Traffic-Sign Classification

The advancement of machine learning (ML) has made autonomous vehicles (AVs) viable, offering benefits in user comfort and traffic efficiency. However, AV safety depends on the performance and reliability of these ML models, which can be compromised by adversarial attacks. In traffic sign classification, such attacks can be carried out by placing crafted stickers on road signs to mislead ML-based image classifiers. This work proposes a hybrid defence system for traffic sign classification under sticker attacks, specifically targeting resource-constrained devices. The system integrates an adversarial detector (AD) subnetwork with the main classifier to detect potential attacks. Upon detection, it relies on auxiliary models constructed using domain knowledge and enhanced with the Defense against Occlusion Attack (DOA) technique. Designed for deployment on low-cost FPGAs, the system uses Binarised Neural Networks (BNNs) for efficient hardware implementation. It achieves 93.82% and 76.50% accuracy on the LISA and GTSRB datasets under strong sticker attacks, showing improvements of up to 70% over the compromised main classifier and over 15% beyond the DOA-only baseline. Hardware evaluations on Zynq-based SoCs (Zedboard and Ultra96) achieve real-time throughput of over 9,000 fps with only $$\sim $$ 2.8 W power consumption, or up to 32% power savings at 100 fps. These results demonstrate the practical feasibility of a robust, lightweight, and real-time defence against sticker attacks suitable for deployment in intelligent vehicle systems.