MCP Failure Lab: Reproducible Tool-Failure Fixtures and Cross-Platform Traces for Testing Agent Reliability

Agents that call tools through the Model Context Protocol (MCP) must handle tools that fail, hang, return malformed results or disconnect. These failures are hard to reproduce on demand, which makes client recovery logic hard to regression-test. We release MCP Failure Lab, a deliberately faulty MCP server built on the official MCP TypeScript SDK (version 2.3.1). It has six injectable fault modes and comes with a bounded reference client and four before/after policy fixtures with explicit acceptance checks. We release one fixture-suite run from each of four platforms: AWS Lambda (ARM64), a Google Cloud Run job (AMD64), a local container and a Turing RK1 arm64 module. The resulting 32 runs are published as a Hugging Face dataset. Each row holds an OpenAI-message-shaped trace, per-attempt timings, a SHA-256 trace hash and heuristic findings from Agent Trace Clinic. All 32 runs reached their expected outcome and attempt count. In each of the eight fixture-variant cells, the trace hash was the same on all four platforms. The release is small and synthetic. No language model participated, one suite per platform is released, and the public files declare cloud origin but do not prove it. We describe the fault surface, the fixture design, the data schema and these limits. The release is intended as regression fixtures for MCP clients and agent harnesses, not as a model leaderboard. Drafted with AI assistance; reviewed and approved by the author. CloudLab Works.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-27
DOI
https://doi.org/10.5281/zenodo.23720941
Primary Topic
Software System Performance and Reliability
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

MCP Failure Lab: Reproducible Tool-Failure Fixtures and Cross-Platform Traces for Testing Agent Reliability

Alex Alvord
Zenodo (CERN European Organization for Nuclear Research)
Software System Performance and Reliability
preprint

MCP Failure Lab: Reproducible Tool-Failure Fixtures and Cross-Platform Traces for Testing Agent Reliability

Alex Alvord
preprint en

Abstract

Agents that call tools through the Model Context Protocol (MCP) must handle tools that fail, hang, return malformed results or disconnect. These failures are hard to reproduce on demand, which makes client recovery logic hard to regression-test. We release MCP Failure Lab, a deliberately faulty MCP server built on the official MCP TypeScript SDK (version 2.3.1). It has six injectable fault modes and comes with a bounded reference client and four before/after policy fixtures with explicit acceptance checks. We release one fixture-suite run from each of four platforms: AWS Lambda (ARM64), a Google Cloud Run job (AMD64), a local container and a Turing RK1 arm64 module. The resulting 32 runs are published as a Hugging Face dataset. Each row holds an OpenAI-message-shaped trace, per-attempt timings, a SHA-256 trace hash and heuristic findings from Agent Trace Clinic. All 32 runs reached their expected outcome and attempt count. In each of the eight fixture-variant cells, the trace hash was the same on all four platforms. The release is small and synthetic. No language model participated, one suite per platform is released, and the public files declare cloud origin but do not prove it. We describe the fault surface, the fixture design, the data schema and these limits. The release is intended as regression fixtures for MCP clients and agent harnesses, not as a model leaderboard. Drafted with AI assistance; reviewed and approved by the author. CloudLab Works.

Zenodo (CERN European Organization for Nuclear Research)
Software System Performance and Reliability
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon