Byte for Byte Identical, 14.327 Times Slower, and Unfinished at Scale: A Pre-registered Measurement of Apache Beam's Portability Tax on the DirectRunner

What does a portable pipeline model cost, and what does it buy? This study pre-registered the question against an existing record. The six batch transformations of an already published Spark and DuckDB study (Track B2 of the data-engineering-scale-lab, closed 2026-11-07) were re-implemented as Apache Beam pipelines in the Python SDK (Apache Beam 2.77.0), run on the DirectRunner on the same machine that produced the comparison record, and scored under rules frozen in a pre-registration before any Beam run existed. Three hypotheses were registered. At the smaller scale (data derived from TPC-H at scale factor 0.1), all six transformations completed: the Beam total was 169.92 s against the Spark local total of 11.86 s, a portability tax of 14.327 times, unevenly spread (per-transformation ratios from 5.831 to 22.137, a spread of 3.796, secondary evidence only). On correctness, Beam outputs matched the comparison record's run-1 SHA-256 checksum on 7 of the 12 registered cells with zero mismatches, and run 5 reproduced run 1 on every matched cell. At scale factor 1, the central join (T2, 6,001,215 output rows) exhausted the machine in nine consecutive passes across the two runner configurations the frozen protocol allowed, at peak process-tree memory between 4,497,064 KB and 5,429,284 KB on a 7.7 GiB shared virtual machine; only T1 ever completed (mean 152.97 s over five timed runs). Under the frozen rules all three verdicts are therefore INCONCLUSIVE: the tax is measured only at the smaller scale (H1, H3), and exact reproduction is proven on 7 of 12 cells, not contradicted on the rest (H2). The claims are deliberately narrow: one machine, one workload, one runner. Within those bounds, the portability is real (identical bytes under a canonical checksum), the premium at small scale is about fourteen times the Spark runtime, and the reference runner has a resource floor that the larger scale crosses.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24328317
Primary Topic
Cloud Computing and Resource Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Byte for Byte Identical, 14.327 Times Slower, and Unfinished at Scale: A Pre-registered Measurement of Apache Beam's Portability Tax on the DirectRunner

Jesun Ahmad Ushno
Zenodo (CERN European Organization for Nuclear Research)
Cloud Computing and Resource Management
preprint

Byte for Byte Identical, 14.327 Times Slower, and Unfinished at Scale: A Pre-registered Measurement of Apache Beam's Portability Tax on the DirectRunner

Jesun Ahmad Ushno
preprint en

Abstract

What does a portable pipeline model cost, and what does it buy? This study pre-registered the question against an existing record. The six batch transformations of an already published Spark and DuckDB study (Track B2 of the data-engineering-scale-lab, closed 2026-11-07) were re-implemented as Apache Beam pipelines in the Python SDK (Apache Beam 2.77.0), run on the DirectRunner on the same machine that produced the comparison record, and scored under rules frozen in a pre-registration before any Beam run existed. Three hypotheses were registered. At the smaller scale (data derived from TPC-H at scale factor 0.1), all six transformations completed: the Beam total was 169.92 s against the Spark local total of 11.86 s, a portability tax of 14.327 times, unevenly spread (per-transformation ratios from 5.831 to 22.137, a spread of 3.796, secondary evidence only). On correctness, Beam outputs matched the comparison record's run-1 SHA-256 checksum on 7 of the 12 registered cells with zero mismatches, and run 5 reproduced run 1 on every matched cell. At scale factor 1, the central join (T2, 6,001,215 output rows) exhausted the machine in nine consecutive passes across the two runner configurations the frozen protocol allowed, at peak process-tree memory between 4,497,064 KB and 5,429,284 KB on a 7.7 GiB shared virtual machine; only T1 ever completed (mean 152.97 s over five timed runs). Under the frozen rules all three verdicts are therefore INCONCLUSIVE: the tax is measured only at the smaller scale (H1, H3), and exact reproduction is proven on 7 of 12 cells, not contradicted on the rest (H2). The claims are deliberately narrow: one machine, one workload, one runner. Within those bounds, the portability is real (identical bytes under a canonical checksum), the premium at small scale is about fourteen times the Spark runtime, and the reference runner has a resource floor that the larger scale crosses.

Zenodo (CERN European Organization for Nuclear Research)
Cloud Computing and Resource Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon