Pre-registered design: energy and cost per query of CPU and GPU dataframe engines (Polars, Spark, cudf-polars, RAPIDS) on A100, H100, GH200 and B200 hosts and a CPU-only bare-metal host (Paper 3), locked 2026-11-16

Pre-registration (locked design) for Paper 3. This record timestamps the locked design file docs/PAPER3_ENERGY_DESIGN.md (git tag paper3-design-lock in the repository below) and the pre-lock probe evidence it cites. No study query had been run when the design was locked; the probes used synthetic loads only (a CUDA spin kernel, nvidia-smi pollers and pinned CPU spin loops). Aim. Rigorous, reproducible evaluation methods and tools that help U.S. organizations choose cost- and energy-efficient CPU or GPU engines for analytical data processing. The study measures energy and cost per query of Polars and Apache Spark, each on the CPU and on the GPU, with scope-tagged energy (NVIDIA NVML energy counter on Lambda GPU hosts; Intel RAPL on a GCP c3-highmem-192-metal CPU-only host), a 10 s window rule with per-host measurement resolution fixed by pre-lock probes, integrity and telemetry gates, a Williams Latin-square order seeded by the lock date, and pre-specified break-even rules. It contributes a measurement protocol and break-even scales, not a vendor ranking. Files. PAPER3_ENERGY_DESIGN.md (the locked design); paper3-prelock-evidence.tar.gz (docs/paper3-evidence/, docs/PAPER3_INFRA.md and paper3_hosts.json at the lock commit); SHA256SUMS. Funding and conflict of interest. Lambda provided $10,000 and AWS $6,000 in compute credits for Paper 3; neither sponsor had any role in design, analysis, interpretation, review or the decision to publish. Use of AI tools. AI coding and writing assistants (Anthropic's Claude via Claude Code; OpenAI Codex in the earlier exploratory work) were used to write and test code, run and monitor the cloud experiments, draft the design and analysis plan, and draft and edit the text. The author directed the study, reviewed and approved the design, the analyses and the text, and takes full responsibility for the content. Workload notice. The TPC-H-derived workload is derived from the TPC-H Benchmark and as such is not comparable to published TPC-H Benchmark results, as the TPC-H-derived workload results do not comply with the TPC-H Benchmark Specification. The energy figures are not TPC-Energy metrics.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24284711
Primary Topic
Parallel Computing and Optimization Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Pre-registered design: energy and cost per query of CPU and GPU dataframe engines (Polars, Spark, cudf-polars, RAPIDS) on A100, H100, GH200 and B200 hosts and a CPU-only bare-metal host (Paper 3), locked 2026-11-16

Daniel Ejiofor
2 links
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
article

Pre-registered design: energy and cost per query of CPU and GPU dataframe engines (Polars, Spark, cudf-polars, RAPIDS) on A100, H100, GH200 and B200 hosts and a CPU-only bare-metal host (Paper 3), locked 2026-11-16

Daniel Ejiofor
article en

Abstract

Pre-registration (locked design) for Paper 3. This record timestamps the locked design file docs/PAPER3_ENERGY_DESIGN.md (git tag paper3-design-lock in the repository below) and the pre-lock probe evidence it cites. No study query had been run when the design was locked; the probes used synthetic loads only (a CUDA spin kernel, nvidia-smi pollers and pinned CPU spin loops). Aim. Rigorous, reproducible evaluation methods and tools that help U.S. organizations choose cost- and energy-efficient CPU or GPU engines for analytical data processing. The study measures energy and cost per query of Polars and Apache Spark, each on the CPU and on the GPU, with scope-tagged energy (NVIDIA NVML energy counter on Lambda GPU hosts; Intel RAPL on a GCP c3-highmem-192-metal CPU-only host), a 10 s window rule with per-host measurement resolution fixed by pre-lock probes, integrity and telemetry gates, a Williams Latin-square order seeded by the lock date, and pre-specified break-even rules. It contributes a measurement protocol and break-even scales, not a vendor ranking. Files. PAPER3_ENERGY_DESIGN.md (the locked design); paper3-prelock-evidence.tar.gz (docs/paper3-evidence/, docs/PAPER3_INFRA.md and paper3_hosts.json at the lock commit); SHA256SUMS. Funding and conflict of interest. Lambda provided $10,000 and AWS $6,000 in compute credits for Paper 3; neither sponsor had any role in design, analysis, interpretation, review or the decision to publish. Use of AI tools. AI coding and writing assistants (Anthropic's Claude via Claude Code; OpenAI Codex in the earlier exploratory work) were used to write and test code, run and monitor the cloud experiments, draft the design and analysis plan, and draft and edit the text. The author directed the study, reviewed and approved the design, the analyses and the text, and takes full responsibility for the content. Workload notice. The TPC-H-derived workload is derived from the TPC-H Benchmark and as such is not comparable to published TPC-H Benchmark results, as the TPC-H-derived workload results do not comply with the TPC-H Benchmark Specification. The energy figures are not TPC-Energy metrics.

Zenodo (CERN European Organization for Nuclear Research)
Boston University (US)
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon