AgentFail: A Stateful Paradigm for Quantifying Autonomous Agent Over-Reliance and Multi-Turn Operational Failures

Abstract As Large Language Models (LLMs) transition from passive text generators to autonomous AI agents executing multi-step workflows, evaluating their operational safety becomes a critical, unsolved bottleneck. Current evaluation frameworks heavily prioritize conversational fluency or static reasoning, completely ignoring the severe financial, legal, and operational risks of autonomous actions in live software environments. To bridge this critical gap, we introduce AgentFail, an open-source, turnkey benchmark engineered to evaluate agentic over-reliance and catastrophic failure modes across leading LLM architectures within Dockerized enterprise software environments (including Salesforce, Jira, and SQL-based ERP networks). Comprising 500 complex, multi-turn corporate task scenarios embedded with subtle operational edge cases and ambiguous constraints, AgentFail exposes a pervasive vulnerability in frontier architectures: current state-of-the-art reasoning models exhibit acute automation over-reliance, confidently executing destructive actions—such as unauthorized data deletion, critical financial ledger miscalculations, and privacy-violating data distributions—in over 42% of empirical trials without triggering built-in safety overrides or initiating human validation requests. We introduce a formal taxonomy of agentic failure alongside two novel continuous evaluation metrics: Error Propagation Velocity (Vep ) and the Over-Reliance Coefficient (Ω). By open-sourcing our entire testing infrastructure, automated environment wrappers, and baseline evaluation suite, we provide a rigorous, highly transferable safety baseline to accelerate the development of alignment strategies and guardrails for next-generation agentic workflows.