NVIDIA TensorRT vs ONNX Runtime: 2 ms p99 SLO at batch size 1 on T4

NVIDIA TensorRT meets the 2 ms p99 latency SLO on T4 at batch size 1, while ONNX Runtime requires its timing cache to pass identical fixed-shape benchmarks. Independent technical note mirroring the canonical version: https://hfrtai.com/blog/nvidia-tensorrt-vs-onnx-runtime-2-ms-p99-slo-at-batch-size-1-on-t4.php Sources consulted: https://www.bluusun.com/

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-29
DOI
https://doi.org/10.5281/zenodo.23792023
Primary Topic
Parallel Computing and Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

NVIDIA TensorRT vs ONNX Runtime: 2 ms p99 SLO at batch size 1 on T4

Owen Gallagher
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
preprint

NVIDIA TensorRT vs ONNX Runtime: 2 ms p99 SLO at batch size 1 on T4

Owen Gallagher
preprint en

Abstract

NVIDIA TensorRT meets the 2 ms p99 latency SLO on T4 at batch size 1, while ONNX Runtime requires its timing cache to pass identical fixed-shape benchmarks. Independent technical note mirroring the canonical version: https://hfrtai.com/blog/nvidia-tensorrt-vs-onnx-runtime-2-ms-p99-slo-at-batch-size-1-on-t4.php Sources consulted: https://www.bluusun.com/

Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
NVIDIA TensorRT vs ONNX Runtime: 2 ms p99 SLO at batch size 1 on T4 — Owen Gallagher · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS