At the Memory Wall, the Hand-Written Kernel Only Ties Eager: A Pre-registered Study of Fused Softmax in Triton against torch.softmax and torch.compile on One Laptop GPU

Writing the kernel yourself is widely assumed to be the fast option by default. This study pre-registered that assumption as a testable claim and measured it on fused row-wise softmax (float32), on one named GPU (NVIDIA GeForce RTX 5050 Laptop GPU, 8 GB), against the two ways the same operation is normally run: torch.softmax eager and torch.compile. The protocol, the shapes, and the verdict rules were frozen in the repository's first content commit, before any measured run. Rows were fixed at 4096; the column ladder was 1024, 2048, 4096, 8192, and 16384, with 16384 columns the verdict size; each arm at each size received one correctness call, 5 warmup calls, and 25 timed calls bracketed by CUDA events, and the scored figure is effective bandwidth (bytes moved over median time). Of the three frozen hypotheses, one is REFUTED and two are CONFIRMED. H1 is REFUTED: the hand-written Triton kernel does not beat eager by the frozen 1.30 times line at the verdict size; it reaches 337.4 GB/s against eager's 338.6 GB/s, a ratio of 0.996. Both kernels sit at the device's memory wall, and at the wall there is no 1.30 times to be had. H2 is CONFIRMED: Triton lands inside the frozen parity band against torch.compile, ratio 1.044 against a band of 0.85 to 1.18, and the compiled arm is the slowest of the three at both of the two largest sizes. H3 is CONFIRMED: the Triton kernel matches the eager reference to a worst-case maximum absolute error of 7.45e-09 across all five sizes, far inside the frozen tolerance of 1e-5. The hand-written kernel is the fastest of the three at 4096 and 8192 columns (326.8 and 340.0 GB/s) and ties eager at the size the protocol judges. The claims are deliberately narrow: one operation, one shape family, one laptop GPU whose clocks were not locked, one pass of the protocol, and a smallest ladder cell that is partly cache-resident and is reported as context, not scored.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24328639
Primary Topic
Parallel Computing and Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

At the Memory Wall, the Hand-Written Kernel Only Ties Eager: A Pre-registered Study of Fused Softmax in Triton against torch.softmax and torch.compile on One Laptop GPU

Jesun Ahmad Ushno
4 links
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
preprint

At the Memory Wall, the Hand-Written Kernel Only Ties Eager: A Pre-registered Study of Fused Softmax in Triton against torch.softmax and torch.compile on One Laptop GPU

Jesun Ahmad Ushno
preprint en

Abstract

Writing the kernel yourself is widely assumed to be the fast option by default. This study pre-registered that assumption as a testable claim and measured it on fused row-wise softmax (float32), on one named GPU (NVIDIA GeForce RTX 5050 Laptop GPU, 8 GB), against the two ways the same operation is normally run: torch.softmax eager and torch.compile. The protocol, the shapes, and the verdict rules were frozen in the repository's first content commit, before any measured run. Rows were fixed at 4096; the column ladder was 1024, 2048, 4096, 8192, and 16384, with 16384 columns the verdict size; each arm at each size received one correctness call, 5 warmup calls, and 25 timed calls bracketed by CUDA events, and the scored figure is effective bandwidth (bytes moved over median time). Of the three frozen hypotheses, one is REFUTED and two are CONFIRMED. H1 is REFUTED: the hand-written Triton kernel does not beat eager by the frozen 1.30 times line at the verdict size; it reaches 337.4 GB/s against eager's 338.6 GB/s, a ratio of 0.996. Both kernels sit at the device's memory wall, and at the wall there is no 1.30 times to be had. H2 is CONFIRMED: Triton lands inside the frozen parity band against torch.compile, ratio 1.044 against a band of 0.85 to 1.18, and the compiled arm is the slowest of the three at both of the two largest sizes. H3 is CONFIRMED: the Triton kernel matches the eager reference to a worst-case maximum absolute error of 7.45e-09 across all five sizes, far inside the frozen tolerance of 1e-5. The hand-written kernel is the fastest of the three at 4096 and 8192 columns (326.8 and 340.0 GB/s) and ties eager at the size the protocol judges. The claims are deliberately narrow: one operation, one shape family, one laptop GPU whose clocks were not locked, one pass of the protocol, and a smallest ladder cell that is partly cache-resident and is reported as context, not scored.

Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon