On the NVIDIA GB10, a GPU copy from a private file mapping duplicates every page in host memory: a PyTorch-only test, a discrete-GPU control, and why model loads need twice their size

Version 2, 19 November 2026: adds the cause and more ways around it. On an NVIDIA DGX Spark (GB10) the CPU and GPU share one 128 GB memory pool. When data in a private, writable file mapping is copied to the GPU, the process keeps a private copy of every page it copied until the mapping is closed, even though nothing writes to those pages. Model loaders map checkpoint files this way (safetensors maps them private and writable, and transformers 5.17 keeps every file open until loading ends), so loading a model needs about twice its size in memory. A 27B model would not load on our unit for this reason. A discrete NVIDIA A40 showed no copies. The cause, from a kernel trace and the driver's open source code. In the copies we traced from a freshly mapped file, CUDA first asked the driver to load the source pages, and the driver loaded them with the same permission the mapping has (UVM_POPULATE_PERMISSIONS_INHERIT in nvidia-uvm/uvm_populate_pageable.c, driver 580.173.02). A mapping that allows writing therefore got a write request on every page, and Linux answers a write request on a private mapping by making a private copy of the page. When the CPU had read every page first, CUDA usually skipped that step: 10 of 13 such copies left no copies, and the other 3 went through it as before. Version 1 said the copies fit the driver faulting or pinning the pages with write intent; the trace shows the faulting and found no pinning. What avoids it, measured with 512 MiB (medians of three runs, with another job using the GPU): mapping the file read only left no copies, and the copy took 131 ms instead of 415 ms. Copying through our own pinned buffers left no copies and took 25 ms. Copying each tensor into RAM first left no copies and took 33 ms; with that change our 27B model loaded in 10 to 11 seconds from the page cache. Reading the file instead of mapping it (safetensors' pread backend, one run) and safetensors' load_file straight to the GPU (three runs) also left no copies. CUDA's own read only option, cudaHostRegisterReadOnly, returned error 801 (not supported) on GB10. The driver already has a mode that loads pages without asking for write access, UVM_POPULATE_PERMISSIONS_ANY, and uses it when the GPU itself reads a page. Using it for copy sources should avoid the copies for every framework; we have not tested a driver changed that way. Included: the note (PDF), the test scripts, their outputs on the Spark and the A40, and the kernel traces.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24358363
Primary Topic
Parallel Computing and Optimization Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

On the NVIDIA GB10, a GPU copy from a private file mapping duplicates every page in host memory: a PyTorch-only test, a discrete-GPU control, and why model loads need twice their size

Mark Hutto
Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
article

On the NVIDIA GB10, a GPU copy from a private file mapping duplicates every page in host memory: a PyTorch-only test, a discrete-GPU control, and why model loads need twice their size

Mark Hutto
article en

Abstract

Version 2, 19 November 2026: adds the cause and more ways around it. On an NVIDIA DGX Spark (GB10) the CPU and GPU share one 128 GB memory pool. When data in a private, writable file mapping is copied to the GPU, the process keeps a private copy of every page it copied until the mapping is closed, even though nothing writes to those pages. Model loaders map checkpoint files this way (safetensors maps them private and writable, and transformers 5.17 keeps every file open until loading ends), so loading a model needs about twice its size in memory. A 27B model would not load on our unit for this reason. A discrete NVIDIA A40 showed no copies. The cause, from a kernel trace and the driver's open source code. In the copies we traced from a freshly mapped file, CUDA first asked the driver to load the source pages, and the driver loaded them with the same permission the mapping has (UVM_POPULATE_PERMISSIONS_INHERIT in nvidia-uvm/uvm_populate_pageable.c, driver 580.173.02). A mapping that allows writing therefore got a write request on every page, and Linux answers a write request on a private mapping by making a private copy of the page. When the CPU had read every page first, CUDA usually skipped that step: 10 of 13 such copies left no copies, and the other 3 went through it as before. Version 1 said the copies fit the driver faulting or pinning the pages with write intent; the trace shows the faulting and found no pinning. What avoids it, measured with 512 MiB (medians of three runs, with another job using the GPU): mapping the file read only left no copies, and the copy took 131 ms instead of 415 ms. Copying through our own pinned buffers left no copies and took 25 ms. Copying each tensor into RAM first left no copies and took 33 ms; with that change our 27B model loaded in 10 to 11 seconds from the page cache. Reading the file instead of mapping it (safetensors' pread backend, one run) and safetensors' load_file straight to the GPU (three runs) also left no copies. CUDA's own read only option, cudaHostRegisterReadOnly, returned error 801 (not supported) on GB10. The driver already has a mode that loads pages without asking for write access, UVM_POPULATE_PERMISSIONS_ANY, and uses it when the GPU itself reads a page. Using it for copy sources should avoid the copies for every framework; we have not tested a driver changed that way. Included: the note (PDF), the test scripts, their outputs on the Spark and the A40, and the kernel traces.

Zenodo (CERN European Organization for Nuclear Research)
Parallel Computing and Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon