When does online vision-language inference help? A survey of socially aware robot navigation

Abstract Vision-language models (VLMs) and large language models are increasingly built into the navigation stacks of robots that share space with people, where the models read cues that occupancy maps omit, such as a raised hand or a conversing group. This survey organizes 41 cases reported from 2024 to 2026, 34 on socially aware navigation and 7 adjacent mechanism extensions, by where the model enters the stack: scene interpretation, cost modulation, subgoal or path proposal, hierarchical coordination, direct action output and offline transfer or generation. For each of these six patterns we describe representative systems and their reported results, and we review the datasets, simulators, benchmarks and metrics behind them. We then discuss what the reported comparisons establish about running the model online, separating system utility, component contribution and online necessity. In these comparisons, cue-aware systems improved the outcomes that hinge on their cue, although whole-system differences limit attribution to the VLM; in one offline study, a model selector helped on hard selections and hurt on routine ones; in one deployment, delayed advice stayed useful when fused with current plans; model-supervised students navigated without runtime teacher queries, but online, cached and distilled alternatives have not been compared with student computation included; and better answers have not yet been linked to better behavior. We also define an invocation ratio (model requests per control decision), propose a minimum reporting set for deployment comparisons, and outline open challenges in real-time reasoning, spatial grounding and safety.