On the NVIDIA GB10, a GPU copy from a private file mapping duplicates every page in host memory: a PyTorch-only test, a discrete-GPU control, and why model loads need twice their size
Version 2, 19 November 2026: adds the cause and more ways around it. On an NVIDIA DGX Spark (GB10) the CPU and GPU share one 128 GB memory pool. When data in a private, writable file mapping is copied to the GPU, the process keeps a private copy of every page it copied until the mapping is closed, even though nothing writes to those pages. Model loaders map checkpoint files this way (safetensors maps them private and writable, and transformers 5.17 keeps every file open until loading ends), so loading a model needs about twice its size in memory. A 27B model would not load on our unit for this reason. A discrete NVIDIA A40 showed no copies. The cause, from a kernel trace and the driver's open source code. In the copies we traced from a freshly mapped file, CUDA first asked the driver to load the source pages, and the driver loaded them with the same permission the mapping has (UVM_POPULATE_PERMISSIONS_INHERIT in nvidia-uvm/uvm_populate_pageable.c, driver 580.173.02). A mapping that allows writing therefore got a write request on every page, and Linux answers a write request on a private mapping by making a private copy of the page. When the CPU had read every page first, CUDA usually skipped that step: 10 of 13 such copies left no copies, and the other 3 went through it as before. Version 1 said the copies fit the driver faulting or pinning the pages with write intent; the trace shows the faulting and found no pinning. What avoids it, measured with 512 MiB (medians of three runs, with another job using the GPU): mapping the file read only left no copies, and the copy took 131 ms instead of 415 ms. Copying through our own pinned buffers left no copies and took 25 ms. Copying each tensor into RAM first left no copies and took 33 ms; with that change our 27B model loaded in 10 to 11 seconds from the page cache. Reading the file instead of mapping it (safetensors' pread backend, one run) and safetensors' load_file straight to the GPU (three runs) also left no copies. CUDA's own read only option, cudaHostRegisterReadOnly, returned error 801 (not supported) on GB10. The driver already has a mode that loads pages without asking for write access, UVM_POPULATE_PERMISSIONS_ANY, and uses it when the GPU itself reads a page. Using it for copy sources should avoid the copies for every framework; we have not tested a driver changed that way. Included: the note (PDF), the test scripts, their outputs on the Spark and the A40, and the kernel traces.
Authors
- Mark Hutto (ORCID: https://orcid.org/0009-0005-7841-6262)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24358363
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00