At the Memory Wall, the Hand-Written Kernel Only Ties Eager: A Pre-registered Study of Fused Softmax in Triton against torch.softmax and torch.compile on One Laptop GPU
Writing the kernel yourself is widely assumed to be the fast option by default. This study pre-registered that assumption as a testable claim and measured it on fused row-wise softmax (float32), on one named GPU (NVIDIA GeForce RTX 5050 Laptop GPU, 8 GB), against the two ways the same operation is normally run: torch.softmax eager and torch.compile. The protocol, the shapes, and the verdict rules were frozen in the repository's first content commit, before any measured run. Rows were fixed at 4096; the column ladder was 1024, 2048, 4096, 8192, and 16384, with 16384 columns the verdict size; each arm at each size received one correctness call, 5 warmup calls, and 25 timed calls bracketed by CUDA events, and the scored figure is effective bandwidth (bytes moved over median time). Of the three frozen hypotheses, one is REFUTED and two are CONFIRMED. H1 is REFUTED: the hand-written Triton kernel does not beat eager by the frozen 1.30 times line at the verdict size; it reaches 337.4 GB/s against eager's 338.6 GB/s, a ratio of 0.996. Both kernels sit at the device's memory wall, and at the wall there is no 1.30 times to be had. H2 is CONFIRMED: Triton lands inside the frozen parity band against torch.compile, ratio 1.044 against a band of 0.85 to 1.18, and the compiled arm is the slowest of the three at both of the two largest sizes. H3 is CONFIRMED: the Triton kernel matches the eager reference to a worst-case maximum absolute error of 7.45e-09 across all five sizes, far inside the frozen tolerance of 1e-5. The hand-written kernel is the fastest of the three at 4096 and 8192 columns (326.8 and 340.0 GB/s) and ties eager at the size the protocol judges. The claims are deliberately narrow: one operation, one shape family, one laptop GPU whose clocks were not locked, one pass of the protocol, and a smallest ladder cell that is partly cache-resident and is reported as context, not scored.
Authors
- Jesun Ahmad Ushno (ORCID: https://orcid.org/0000-0002-6070-9025)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24328639
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- preprint