Ferrule: Bounded Storage-Backed Weight Residency
Storage-backed weight residency requires more than a small cache: reads, transfers, and device consumers can retain resources after a request or binding is logically finished. We present Ferrule, an architecture that connects source-bound materialization plans, stage-local admission, exact completion identity, and ownership through retirement. Its implemented expert path combines a persistent bounded read worker, optional next-expert lookahead, and grouped synchronous uploads without changing the synchronous forward interface. We evaluate a single-device, text-only Qwen3.5-35B-A3B-FP8 profile using BF16 execution weights and explicit F32 boundaries. Separate public-CLI acceptance checks complete 48 requests; a transfer-policy pilot completes 96 requests across eight fresh servers, including 40 measured, output-checked requests. Grouped uploads improve observed client latency and phase-qualified goodput with lookahead, but increase time to first content with blocking reads while end-to-end latency remains nearly unchanged. With two pairs per read schedule, these are conditional, descriptive benefits rather than a universal performance result. An earlier five-pair study separately isolates the effect of read scheduling. Quality gates, selected-process profiles, component faults, and HTTP cancellation provide complementary evidence about materialization and custody. The measurements bound selected logical host and cache accounts, not physical RSS, storage traffic, or universal device release; integrated five-class conservation and end-to-end fault coverage remain unverified.
Authors
- Huanbing Lu (ORCID: https://orcid.org/0000-0002-5050-4851)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-20
- DOI
- https://doi.org/10.5281/zenodo.24397260
- Primary Topic
- Software System Performance and Reliability
- Type
- preprint