On the Nature of AI Agent Failures on DeepSeek V4 Pro and Gemini 3.8 Flash

This document presents an analytical breakdown of the behavior of AI agents that call functions with real effects. The agents operated on the DeepSeek V4 Pro and Gemini 3.8 Flash models. The "Warehouse" agent testing system is built on the methodology whose main provisions we set out previously (doi.org, habr.com). Subsequent innovations concerned primarily the optimization of the method, the expansion of the testing system’s analytical capabilities, and the improvement of the interface and visualization of conclusions in run reports, while the main theoretical provisions remained unchanged. To verify whether an AI agent will call a function that performs an action with a real effect when it must not be called because the unstated conditions of its application are not met, and to compare the action performed with the response the agent gives to the user. The conclusions concern the character of failures in upholding unstated conditions. In particular, it is shown that DeepSeek works with a condition as an object of reasoning, whereas Gemini treats it as a signal; an analysis is provided of which of the identified layers (communicative, logical, or cognitive) the failures occur in for each model; and the agent’s reporting to the user as well as its responses to a persistent remark from the operator are examined.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-12
DOI
https://doi.org/10.5281/zenodo.24159105
Primary Topic
Artificial Intelligence Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

On the Nature of AI Agent Failures on DeepSeek V4 Pro and Gemini 3.8 Flash

ОЛЬГА ПЕТРОВНА СКИДАН, Олег Викторович Мацканюк
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence Applications
article

On the Nature of AI Agent Failures on DeepSeek V4 Pro and Gemini 3.8 Flash

ОЛЬГА ПЕТРОВНА СКИДАН, Олег Викторович Мацканюк
article en

Abstract

This document presents an analytical breakdown of the behavior of AI agents that call functions with real effects. The agents operated on the DeepSeek V4 Pro and Gemini 3.8 Flash models. The "Warehouse" agent testing system is built on the methodology whose main provisions we set out previously (doi.org, habr.com). Subsequent innovations concerned primarily the optimization of the method, the expansion of the testing system’s analytical capabilities, and the improvement of the interface and visualization of conclusions in run reports, while the main theoretical provisions remained unchanged. To verify whether an AI agent will call a function that performs an action with a real effect when it must not be called because the unstated conditions of its application are not met, and to compare the action performed with the response the agent gives to the user. The conclusions concern the character of failures in upholding unstated conditions. In particular, it is shown that DeepSeek works with a condition as an object of reasoning, whereas Gemini treats it as a signal; an analysis is provided of which of the identified layers (communicative, logical, or cognitive) the failures occur in for each model; and the agent’s reporting to the user as well as its responses to a persistent remark from the operator are examined.

Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon