Noise-Aware Closed-Loop Domain Randomized Control of Cloud-Based Autonomous Agents

Autonomous agents powered by large language models (LLMs) perform well in simulated environments but struggle under real-world uncertainties. This paper introduces a noise-aware, closed-loop domain randomization framework to evaluate and improve the robustness of cloud-based embodied agents. Our method injects controlled noise-such as sensor errors, command distortions, and object displacements-into the VirtualHome simulator, forcing agents to recover using real-time feedback and replanning. We test five LLM-based agents on 88 tasks under clean and noisy settings. Results show a consistent drop in executability ($4-7 \%$) under noise, revealing hidden vulnerabilities. However, closed-loop control allows agents to recover many failures, maintaining reasonable task performance. The framework highlights gaps in current evaluation practices and provides a more realistic measure of agent reliability. These findings offer a path toward developing more adaptable and deployable AI agents.