Do LLM Agents Need Tool Schemas in Every Session? Cache-Aware Tool Use in Multi-Tenant LLM Systems

Multi-tenant LLM applications often prepend user- or tenant-specific context before large tool schemas. Under longest-prefix prompt caching — the dominant cache discipline in modern LLM serving — this ordering fragments cache reuse: the schema block may be byte-identical across tenants but still fails to share cache entries because divergence occurs before the schema begins. This report studies tool serialization as a cache-economics problem: how prompt-construction choices determine whether large schema blocks amortize across tenants or are repeatedly billed. We define six tool-serialization architectures — naive direct injection (A), cache-aware direct injection (A′), provider-native tool APIs (A_native), top-m schema retrieval (B), goal-delegation broker (D), and broker with internal retrieval (D_rag) — and provide preliminary measurements for three of them (A, B, D) on DeepSeek across K ∈ {5, 10, 50, 200} personalized tenants over 200 tools and 3 sequential calls per tenant. Delegation maintains a 98.4% prompt-cache hit rate independent of K; naive direct injection plateaus at 74% and incurs a 2.10× higher total cost at K=200. We further derive a closed-form cost model parameterized by tenant count K, schema size S, personalization size P, cached-token discount α, tool-use rate q, and per-tenant catalog overlap μ, and prove a cache-geometry theorem characterizing when schema-region miss cost is Θ(S) versus Θ(K·S). Under per-tenant tool subsets — the prevailing real-world case in multi-tenant SaaS — cache-aware direct injection cannot preserve Θ(S) cache geometry in tenant count, while goal-delegation with a unified broker catalog plus runtime ACL can. The paper also contributes a cross-provider cache-behavior evaluation framework comparing Anthropic, OpenAI, DeepSeek, and self-hosted vLLM, and a decision framework mapping deployment characteristics to recommended architecture. Seven falsifiability conditions are pre-registered. This work extends the substrate-thesis introduced in Skill as Memory, Not Document (Sarkar 2026, doi:10.5281/zenodo.20128887). The present report is preliminary; cross-provider replication, BFCL accuracy evaluation, and per-tenant subset sweeps are in progress.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-05-17
DOI
https://doi.org/10.5281/zenodo.20264013
Primary Topic
Advanced Data Storage Technologies
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Do LLM Agents Need Tool Schemas in Every Session? Cache-Aware Tool Use in Multi-Tenant LLM Systems

Pranab Sarkar
Zenodo (CERN European Organization for Nuclear Research)
Advanced Data Storage Technologies
preprint

Do LLM Agents Need Tool Schemas in Every Session? Cache-Aware Tool Use in Multi-Tenant LLM Systems

Pranab Sarkar
preprint en

Abstract

Multi-tenant LLM applications often prepend user- or tenant-specific context before large tool schemas. Under longest-prefix prompt caching — the dominant cache discipline in modern LLM serving — this ordering fragments cache reuse: the schema block may be byte-identical across tenants but still fails to share cache entries because divergence occurs before the schema begins. This report studies tool serialization as a cache-economics problem: how prompt-construction choices determine whether large schema blocks amortize across tenants or are repeatedly billed. We define six tool-serialization architectures — naive direct injection (A), cache-aware direct injection (A′), provider-native tool APIs (A_native), top-m schema retrieval (B), goal-delegation broker (D), and broker with internal retrieval (D_rag) — and provide preliminary measurements for three of them (A, B, D) on DeepSeek across K ∈ {5, 10, 50, 200} personalized tenants over 200 tools and 3 sequential calls per tenant. Delegation maintains a 98.4% prompt-cache hit rate independent of K; naive direct injection plateaus at 74% and incurs a 2.10× higher total cost at K=200. We further derive a closed-form cost model parameterized by tenant count K, schema size S, personalization size P, cached-token discount α, tool-use rate q, and per-tenant catalog overlap μ, and prove a cache-geometry theorem characterizing when schema-region miss cost is Θ(S) versus Θ(K·S). Under per-tenant tool subsets — the prevailing real-world case in multi-tenant SaaS — cache-aware direct injection cannot preserve Θ(S) cache geometry in tenant count, while goal-delegation with a unified broker catalog plus runtime ACL can. The paper also contributes a cross-provider cache-behavior evaluation framework comparing Anthropic, OpenAI, DeepSeek, and self-hosted vLLM, and a decision framework mapping deployment characteristics to recommended architecture. Seven falsifiability conditions are pre-registered. This work extends the substrate-thesis introduced in Skill as Memory, Not Document (Sarkar 2026, doi:10.5281/zenodo.20128887). The present report is preliminary; cross-provider replication, BFCL accuracy evaluation, and per-tenant subset sweeps are in progress.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Advanced Data Storage Technologies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.