Lightweight Surveillance Image Classification Through Hardware-Software Co-Design

This work designs and implements a custom hardware accelerator for single object classification from drone imagery, for surveillance applications. A lightweight attention-based convolutional neural network (CNN) is developed and translated into hardware implementation as an IP/core. This accelerator is implemented as programmable logic (PL) and further optimized with buffer incorporation through high-level synthesis (HLS). The optimized PL is integrated with a processing system (PS), i.e., ZYNQ UltraScale+ MPSoC, enabling a hardware/software co-design paradigm for enhancing the one-versus-rest classification task. The system architecture is tested through the PYNQ overlay process. Experimental results show the architecture is lightweight (1.97 MB) and requires 8.9 million trainable parameters, on the targeted ZCU104 embedded FPGA board. It performs 53.8 Giga MAC operations, achieves an inference time of 1.05 ms, and a throughput of 947.2 frames per second (FPS). It consumes 5.65 W of power at 100 MHz of frequency, shows an efficiency of 9.52 GOPs/W, and dissipates 0.006 J of energy per inference. Codes and subsequent files are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://shorturl.at/iX0jw</uri>.