Reinforcement learning enables autonomous microrobot navigation and intervention in simulated blood capillaries

Abstract Autonomous microrobots navigating biological vasculature could enable targeted drug delivery and thrombolysis, yet training control policies for physiologically complex environments remains an open challenge. Prior reinforcement learning (RL) studies of microrobotic navigation have largely used idealized geometries lacking realistic flow in confined branching, and dense cellular obstacles. Here, we develop a physically grounded, deliberately idealized two-dimensional capillary simulation, with resolved flow fields, explicit red blood cell dynamics, and schematic-derived branching geometry, training RL agents via an advection-dominated concentration field driven by physiological flow speeds. Mapping success across robot size and swimming speed, we show via zero-shot transfer that most training failures reflect optimization limits, not physical ones. A policy transferred from a more demanding condition succeeds far beyond where per-combination training converges. Its eventual collapse offers a tentative lower bound on what is physically achievable. Successful agents converge on Run-and-Rotate strategies. The transferred policy, a variant of this family, shows a distinctive profile, that we believe is responsible for its exceptional transfer success. Without retraining, these agents perform targeted blocking and unblocking of individual vessel branches, restoring red blood cell passage toward baseline levels and suggesting RL as a viable framework for microrobotic intervention in biological networks.