CoreCast: Operator-Level GEMM-Path Compatibility for Low-Precision Inference on NVIDIA V100
Modern inference representations increasingly include bfloat16, FP8, packed FP4, and microscaled formats, while deployed accelerators often predate their native execution paths. This paper evaluates CoreCast as an operator-level GEMM compatibility contract for NVIDIA V100. Rather than proposing a new GEMM, CoreCast unifies format descriptors, scale-metadata restoration, caller-selected range control, output finalization, and explicit failure/policy boundaries around the mature FP16 Tensor Core backend. This distinguishes it from a one-off manual cast, a predecoded cache whose conversion is moved outside the timed call, and an opaque framework fallback. Repeated square-4096 tests measure 67.591 TFLOPS for direct BF16 CoreCast, 10.214 TFLOPS for native PyTorch BF16, and 70.873 TFLOPS for the stronger manual BF16-to-FP16 baseline; the latter confirms that the contribution is path orchestration and format semantics, not a faster matrix kernel. Runtime restoration of the tested FP8/FP4/MX operator payloads reaches 80%--92% of matched predecoded FP16 throughput at square-4096. Recorded application case studies are mixed: BF16 language-model prefill improves while batch-one decode slows slightly, and single-run ComfyUI outcomes depend on model and cast policy. Measured energy claims are limited to task-level board joules per token for the reported language-model phases. Numerical sweeps further motivate an explicit runtime policy that rejects non-finite or invalid metadata, uses direct mapping only after a range check, and otherwise selects range control or fallback. Model-level FP4/NVFP4 language-model execution is not evaluated. The resulting claim is deliberately narrow: a unified operator contract can recover useful V100 GEMM paths for supported low-precision representations, but shape, range, metadata, and application policy determine end-to-end value.
Authors
- Xing Kangwei
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-06-14
- DOI
- https://doi.org/10.5281/zenodo.20687192
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- preprint