A 28-nm Real-Time Reinforcement Learning Processor for Mapless Autonomous Navigation with Unified Actor-Critic Network and Inference-on-Request Scheduling

Autonomous navigation represents the task of navigating between points without human intervention. In many real-world scenarios, the maps are unavailable or unreliable, necessitating robust mapless navigation techniques that rely solely on local observations and target position information without access to global map data [1]. Deep Reinforcement Learning (DRL) has emerged as a viable solution for complex tasks such as motion control [2], game playing [3], and autonomous driving [4], and it is particularly effective for mapless navigation since it allows for systems to adapt to new environments without relying on predefined maps [5]. Traditional DRL approaches such as the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm [6] use scalar Q values to represent the expected reward for specific observation-action pairs [7]. However, those Q values may not effectively capture the uncertainty of the value function, undermining reliability when applied to autonomous navigation [8]. Contrarily, distributional reinforcement learning such as the Distributed Distributional Deep Deterministic Policy Gradient (D4PG) algorithm [9], which models Q values as distributions, provides a more comprehensive understanding of reward structures and is commonly employed in navigation tasks due to its superior performance [10]–[12].