CPU Dataframes versus GPU Acceleration on a Single Host: A Pre-registered, Randomized Comparison of Polars and Spark on a TPC-H-Derived Workload, Replicated across GPU Generations

We compare Polars and Spark, each on the CPU and on the GPU (both GPU paths use NVIDIA cuDF), on one NVIDIA H100 host. The workload is TPC-H-derived (six adapted TPC-H queries, SF1–SF300; six complex queries, SF30–SF300), not a TPC-H benchmark result. Its pre-registered design uses four rounds in a Williams Latin square, timing-independent integrity gates and 95% intervals across rounds. At SF300, GPU speedups over the same framework’s CPU engine were 2.81× (95% CI 2.64–2.99) for Polars and 2.82× (95% CI 2.77–2.88) for Spark on the baseline suite, and 4.70× (95% CI 4.47–4.94) and 4.02× (95% CI 3.77–4.28) on the complex suite. The best GPU engine was faster than Polars CPU from SF10 (baseline) and SF30 (complex), and slower at SF1. Polars GPU rejected 2 of the six complex queries as unsupported (CPU fallback disabled by design). However, the GPU engine requires Polars 1.44.x, and Polars 2.0.0 on the CPU was faster than 1.44.2 at every scale in both suites (all 95% CIs below 1; time ratio 0.73 to 0.92 on the baseline suite and 0.51 to 0.52 on the complex suite). In an exploratory, post-hoc comparison with Polars 2.0.0 (E1), the Polars GPU advantage shrank but stayed above 1 from SF10 (baseline). Campaigns on A100, GH200 and B200 showed the same pattern, with GPU time ratios to the H100 of 1.34 to 1.71 on the A100 (slower) and 0.86 to 1.23 on the GH200 and B200. Across 4 hosts, 0 of 8,832 output comparisons disagreed with Polars CPU. Workload notice. The workload is derived from the TPC-H Benchmark and as such is not comparable to published TPC-H Benchmark results, as the results of this workload do not comply with the TPC-H Benchmark Specification.