CoreCast: Operator-Level GEMM-Path Compatibility for Low-Precision Inference on NVIDIA V100

Modern inference representations increasingly include bfloat16, FP8, packed FP4, and microscaled formats, while deployed accelerators often predate their native execution paths. This paper evaluates CoreCast as an operator-level GEMM compatibility contract for NVIDIA V100. Rather than proposing a new GEMM, CoreCast unifies format descriptors, scale-metadata restoration, caller-selected range control, output finalization, and explicit failure/policy boundaries around the mature FP16 Tensor Core backend. This distinguishes it from a one-off manual cast, a predecoded cache whose conversion is moved outside the timed call, and an opaque framework fallback. Repeated square-4096 tests measure 67.591 TFLOPS for direct BF16 CoreCast, 10.214 TFLOPS for native PyTorch BF16, and 70.873 TFLOPS for the stronger manual BF16-to-FP16 baseline; the latter confirms that the contribution is path orchestration and format semantics, not a faster matrix kernel. Runtime restoration of the tested FP8/FP4/MX operator payloads reaches 80%--92% of matched predecoded FP16 throughput at square-4096. Recorded application case studies are mixed: BF16 language-model prefill improves while batch-one decode slows slightly, and single-run ComfyUI outcomes depend on model and cast policy. Measured energy claims are limited to task-level board joules per token for the reported language-model phases. Numerical sweeps further motivate an explicit runtime policy that rejects non-finite or invalid metadata, uses direct mapping only after a range check, and otherwise selects range control or fallback. Model-level FP4/NVFP4 language-model execution is not evaluated. The resulting claim is deliberately narrow: a unified operator contract can recover useful V100 GEMM paths for supported low-precision representations, but shape, range, metadata, and application policy determine end-to-end value.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-14
DOI
https://doi.org/10.5281/zenodo.20687192
Primary Topic
Parallel Computing and Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

CoreCast: Operator-Level GEMM-Path Compatibility for Low-Precision Inference on NVIDIA V100

Xing Kangwei
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
preprint

CoreCast: Operator-Level GEMM-Path Compatibility for Low-Precision Inference on NVIDIA V100

Xing Kangwei
preprint en

Abstract

Modern inference representations increasingly include bfloat16, FP8, packed FP4, and microscaled formats, while deployed accelerators often predate their native execution paths. This paper evaluates CoreCast as an operator-level GEMM compatibility contract for NVIDIA V100. Rather than proposing a new GEMM, CoreCast unifies format descriptors, scale-metadata restoration, caller-selected range control, output finalization, and explicit failure/policy boundaries around the mature FP16 Tensor Core backend. This distinguishes it from a one-off manual cast, a predecoded cache whose conversion is moved outside the timed call, and an opaque framework fallback. Repeated square-4096 tests measure 67.591 TFLOPS for direct BF16 CoreCast, 10.214 TFLOPS for native PyTorch BF16, and 70.873 TFLOPS for the stronger manual BF16-to-FP16 baseline; the latter confirms that the contribution is path orchestration and format semantics, not a faster matrix kernel. Runtime restoration of the tested FP8/FP4/MX operator payloads reaches 80%--92% of matched predecoded FP16 throughput at square-4096. Recorded application case studies are mixed: BF16 language-model prefill improves while batch-one decode slows slightly, and single-run ComfyUI outcomes depend on model and cast policy. Measured energy claims are limited to task-level board joules per token for the reported language-model phases. Numerical sweeps further motivate an explicit runtime policy that rejects non-finite or invalid metadata, uses direct mapping only after a range check, and otherwise selects range control or fallback. Model-level FP4/NVFP4 language-model execution is not evaluated. The resulting claim is deliberately narrow: a unified operator contract can recover useful V100 GEMM paths for supported low-precision representations, but shape, range, metadata, and application policy determine end-to-end value.

Zenodo (CERN European Organization for Nuclear Research)
Affordable and clean energy
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

CoreCast: Operator-Level GEMM-Path Compatibility for Low-Precision Inference on NVIDIA V100 — Xing Kangwei · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS