YOLOv5n Edge Accelerator Design and Energy Efficiency Optimization Through RISC-V and FPGA Collaboration

Since its inception, the open-source Reduced Instruction Set Computer Five (RISC-V) instruction set architecture has rapidly gained traction due to its modularity, extensibility and customizability, making it a compelling alternative to x86 and ARM in neural network research. However, real-time applications of neural networks remain challenged by limited computing power and low energy efficiency. To address this, this paper propose a RISC-V + Field-Programmable Gate Array (FPGA) collaborative edge accelerator tailored for You Only Look Once version 5 nano (YOLOv5n), aiming to balance performance and resource efficiency. Firstly, this paper leverage the RV64IMAFDC instruction set of the RISC-V to execute finegrained instruction scheduling, enabling efficient preprocessing of input images and enhancing logic computation capabilities. Secondly, this paper restructure the YOLOv5n architecture for FPGA deployment, applying layer fusion techniques to combine activation functions, batch normalization and convolution layers. This significantly reduces computation steps and resource consumption, improving deployment efficiency and hardware adaptability. Thirdly, this paper designed a dual-mode convolution accelerator featuring a reconfigurable Processing Element (PE) array capable of executing both <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1 \times 1$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$3 \times 3$</tex> convolutions simultaneously. By leveraging double buffering and local Random Access Memory (RAM) storage, this design enhances data transmission efficiency while minimizing global memory access, thereby boosting overall throughput during inference. Experimental results on the PolarFire platform validate the accelerator's effectiveness. It achieves 20.4 FPS while consuming only 59,930 LUTs and 164 DSPs, with an energy efficiency of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{2 8. 6 4} \mathbf{~ G O P s} / \mathbf{W}$</tex>. These results highlight the proposed accelerator's potential in enabling real-time, low-power object detection at the edge, particularly in resource-constrained environments.