From Tool Selection to Autonomous Execution: Process Templates, Sandboxed Code, and Modular Orchestration for Reliable LLM Agents at Scale

Large Language Model (LLM) agents are increasingly deployed to select and execute tools from vast libraries autonomously. Recent work identified a phase transition in tool-selection accuracy beyond ~60 tools, driven by semantic confusability. Still, two critical gaps remain: whether description quality can delay this collapse, and whether correctly selected tools are executed reliably under context pressure. We present a comprehensive empirical study that addresses both gaps. In Phase 1 (Routing), we benchmark six models on libraries of up to 2,000 tools and show that expert-authored disambiguation improves routing accuracy by up to 10–20 pp, though the effect is model-dependent. In Phase 2 (Execution), we introduce a novel benchmark isolating execution fidelity under scaling. We uncover schema collapse (73% of bare-prompt outputs missing required keys) and arithmetic collapse (math errors even with correct formulas). Embedding expert process templates (SOPs) eliminates schema failures, while sandboxed code execution removes the arithmetic ceiling, raising end-to-end (E2E) success from 25% to >90% on atomic tasks. We then push the system to its limits: under escalating linguistic noise (typos to adversarial injection) we map a precise fracture point; in a 2x2 query-variation matrix, semantic ambiguity combined with high phrasing novelty causes routing accuracy to drop to 70%; and on a fully autonomous multi-step pipeline with dynamic tool selection supported by a script-integrity guardrail and refined tool filtering we achieve 97.5% E2E success in a rigorous 40-run academic validation sweep. We further ground the phase transition in a semantic packing analysis, identifying a critical cosine similarity threshold (~0.55) beyond which routing collapses, and prove that a context-bounded sub-graph architecture keeps similarity below this danger zone, providing an empirical scalability guarantee. Finally, we embody the full framework in a closed-loop, multi-agent prototype—the Self-Healing Quant Desk—that autonomously detects alpha decay, triggers a researcher agent to rebuild models, and safely redeploys through a trader agent, all guarded by dynamic dataset-aware validation and multiple-testing corrections. A 500-task orchestrator sweep across three professional roles demonstrates 100% E2E completion with partitioned SOPs, while flat baselines decay to 78% and require twice the retries. These findings establish a zero-training, template-driven recipe for building reliable, self-healing agent systems that can scale to thousands of skills.