Low Complexity and Scalable Architecture of Dual-Mode Training and Inference Hardware Accelerator for Deep Q-Network (DQN) Based Edge Computing

Reinforcement Learning (RL) applications are rapidly expanding across various fields, including robotics, autonomous vehicles, drones, communication systems, and control. However, RL computations are inherently complex and resource-intensive due to dynamic state spaces and real-time processing demands. This paper proposes a novel RL accelerator hardware architecture for edge computing, capable of supporting both inference and training modes. The design, tailored for Deep Q-Network (DQN) with Q-values computed using an Artificial Neural Network (ANN), employs hardware sharing and approximate activation functions and multipliers to enhance efficiency. Shared processing modules minimize chip size and power consumption, achieving an area efficiency of 44.25 KIPS/1KLUTs and 14.65 KIPS/1KLUTs. Implementation results demonstrate state processing times of 1.43 μs for inference and 4.31 μs for training. Deployed on an Ultra96 FPGA at 70 MHz, the system delivers 671× and 287× speedups for inference and training, respectively, with a DQN accelerator power consumption of just 13 mW. This architecture is highly suitable for autonomous systems in edge computing environments, addressing resource limitations and enhancing computational efficiency.