Do AI Agents Check Their Work? A Comparative Evaluation of Five Models on Salesforce CRM Tasks

This study evaluates five AI models as agents performing customer relationship management (CRM) tasks in a live Salesforce environment. Each model worked through 50 tasks in five levels of increasing difficulty: (1) everyday requests, (2) multi-turn conversations with a simulated user, (3) tasks where Salesforce silently changes other fields, (4) long multi-step jobs and (5) deliberately faulty tools. Each model made 163 graded attempts at 46 tasks spread across the levels, with most tasks attempted five times. Each attempt was graded on what actually changed in Salesforce, not on the model's account of its work. Claude Opus 5.5 achieved the highest pass rate (142 of 163, 87%), followed by Claude Sonnet 5.5 (139, 85%), DeepSeek V4.1 Flash (133, 82%) and Claude Haiku 5.5 (130, 80%). GPT-6.1 Sol had the lowest pass rate (118, 72%). Only three score gaps were statistically significant: Opus and Sonnet over GPT-6.1 Sol, and Sonnet over Haiku. All others, including Opus over Sonnet, may be due to chance. Cost varied far more than performance. Opus cost 8.4 cents per run and DeepSeek 0.11 cents, about 75 times less, for 9 fewer passes out of 163. All five models completed every routine request and every long job (level 4 tasks) correctly. Their results diverged on three behaviors: re-checking a record after changing it, retrying after a temporary error, and detecting a save that failed without an error message. Every model reported at least 15 of 20 silently failed saves as completed. Opus was the only model that reliably caught the failure, and only when it re-checked a bulk update.