No Abstraction Tax at the Direct Surface: A Pre-registered Paired Study of LangChain Used Directly versus a Native Loop, in Which the Registered Tax Hypothesis Was Refuted and Exactly-Once Recovery Held a Third Time

Most teams meet LangChain before they meet any graph framework, and they use it directly: chains and runnables, no graph layer. Whether that direct surface costs anything against a plain hand-written loop, on identical work, is an empirical question that is usually settled by fashion. This study pre-registered the measurement as the paired sequel to a LangGraph versus native study on the same suite: the same 12 tasks, the same deterministic scripted policy, the same tools, the same step executor, the same budgets, paired by seed, with the native arm imported verbatim from the sibling repository and hash-gated so the control cannot drift. The LangChain arm sequences each step as one invocation of a RunnableSequence of three RunnableLambda stages (load state, run one executor step, persist state) from langchain-core 1.7.6; because langchain-core ships no checkpointer, the persist stage writes the full state dict to the arm's own SQLite store after every step, the same commit-per-step cadence as the native event log. The scripted policy is a declared scope limit: the study measures the orchestration layer only. On task success the arms are equivalent with evidence: overall pass^1 is 0.979 in both arms (94 of 96 runs each), all 12 paired per-task differences are exactly 0.000, and the 90 percent interval for the mean paired difference is [0.000, 0.000], inside the registered margin of plus or minus 0.05 (H1: CONFIRMED). Consistency (pass^k, k = 1 to 8) is identical in the two arms at every k. On cost, the registered abstraction-tax hypothesis fails: the median wall ratio (LangChain over native) is 0.459, under the frozen 1.00 no-tax line, so H2 is REFUTED. LangChain direct was faster than this native implementation on this suite (median 0.398 seconds per run against 0.869; faster in 96 of 96 paired runs, paired bootstrap 95 percent interval for the ratio [0.414, 0.497]), a comparison the paper flags as implementation-sensitive because the native arm commits state synchronously after every step. Policy invocations are identical (median 5.0 per run in both arms). On recovery after a SIGKILL (48 fault runs per arm, two kill classes), both arms pass every run with every intended side effect applied exactly once and zero duplicated effects (H3: CONFIRMED), and the recovery price is also identical: median recomputed steps 0.5 in both arms, with no recomputation in the step-boundary kill class and exactly one recomputed step in the mid-step class, in all runs of both arms. The LangChain arm's durable state store is the smallest of the three arms now measured on this suite (median 8,192 bytes per run, against 16,384 for the native event log and 36,864 for the sibling study's LangGraph checkpoint store), and its arm-specific code is 95 code lines against the native arm's 96. Exactly-once came from the shared idempotency discipline for the third time (native, LangGraph, now LangChain direct), not from the orchestrator.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24329031
Primary Topic
Scientific Computing and Data Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

No Abstraction Tax at the Direct Surface: A Pre-registered Paired Study of LangChain Used Directly versus a Native Loop, in Which the Registered Tax Hypothesis Was Refuted and Exactly-Once Recovery Held a Third Time

Jesun Ahmad Ushno
5 links
Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
preprint

No Abstraction Tax at the Direct Surface: A Pre-registered Paired Study of LangChain Used Directly versus a Native Loop, in Which the Registered Tax Hypothesis Was Refuted and Exactly-Once Recovery Held a Third Time

Jesun Ahmad Ushno
preprint en

Abstract

Most teams meet LangChain before they meet any graph framework, and they use it directly: chains and runnables, no graph layer. Whether that direct surface costs anything against a plain hand-written loop, on identical work, is an empirical question that is usually settled by fashion. This study pre-registered the measurement as the paired sequel to a LangGraph versus native study on the same suite: the same 12 tasks, the same deterministic scripted policy, the same tools, the same step executor, the same budgets, paired by seed, with the native arm imported verbatim from the sibling repository and hash-gated so the control cannot drift. The LangChain arm sequences each step as one invocation of a RunnableSequence of three RunnableLambda stages (load state, run one executor step, persist state) from langchain-core 1.7.6; because langchain-core ships no checkpointer, the persist stage writes the full state dict to the arm's own SQLite store after every step, the same commit-per-step cadence as the native event log. The scripted policy is a declared scope limit: the study measures the orchestration layer only. On task success the arms are equivalent with evidence: overall pass^1 is 0.979 in both arms (94 of 96 runs each), all 12 paired per-task differences are exactly 0.000, and the 90 percent interval for the mean paired difference is [0.000, 0.000], inside the registered margin of plus or minus 0.05 (H1: CONFIRMED). Consistency (pass^k, k = 1 to 8) is identical in the two arms at every k. On cost, the registered abstraction-tax hypothesis fails: the median wall ratio (LangChain over native) is 0.459, under the frozen 1.00 no-tax line, so H2 is REFUTED. LangChain direct was faster than this native implementation on this suite (median 0.398 seconds per run against 0.869; faster in 96 of 96 paired runs, paired bootstrap 95 percent interval for the ratio [0.414, 0.497]), a comparison the paper flags as implementation-sensitive because the native arm commits state synchronously after every step. Policy invocations are identical (median 5.0 per run in both arms). On recovery after a SIGKILL (48 fault runs per arm, two kill classes), both arms pass every run with every intended side effect applied exactly once and zero duplicated effects (H3: CONFIRMED), and the recovery price is also identical: median recomputed steps 0.5 in both arms, with no recomputation in the step-boundary kill class and exactly one recomputed step in the mid-step class, in all runs of both arms. The LangChain arm's durable state store is the smallest of the three arms now measured on this suite (median 8,192 bytes per run, against 16,384 for the native event log and 36,864 for the sibling study's LangGraph checkpoint store), and its arm-specific code is 95 code lines against the native arm's 96. Exactly-once came from the shared idempotency discipline for the third time (native, LangGraph, now LangChain direct), not from the orchestrator.

Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon