DMMU: SmartNIC-Based On-Device MMU for Scalable Virtualized I/O and Memory Overcommitment

Cloud data centers increasingly rely on device virtualization to share high-bandwidth network adapters and accelerators among virtual machines (VMs). Single Root I/O Virtualization (SR-IOV) exposes Virtual Functions (VFs) directly to guest VMs, but the Input/Output Memory Management Unit (IOMMU) on the host processor becomes a bottleneck as the number of VFs grows: its small translation lookaside buffer (TLB) causes severe throughput degradation. Meanwhile, because the hypervisor cannot determine which guest memory regions are used for Direct Memory Access (DMA), it must pin the entire VM memory, preventing memory overcommitment and wasting physical resources. Existing solutions, including Address Translation Services (ATS), the Page Request Interface (PRI), and software-emulated virtual IOMMUs, suffer from compatibility, deployment, or CPU overhead limitations. We propose DMMU, a Device MMU system implemented on programmable SmartNICs that addresses both the IOMMU translation bottleneck and the memory-pinning barrier to overcommitment, with minimal host software modification and zero guest awareness. DMMU introduces two techniques: combo-DMA , which dynamically routes address translations between the IOMMU and an on-device translation engine depending on IOMMU load, and direct-PRI , which enables the SmartNIC to issue page fault requests directly to a host-side driver, bypassing the multi-layer emulation path of conventional PRI in virtualized environments. We implement DMMU through a microcode update to a commodity dual-port 100G NIC and evaluate it on Intel Skylake and Ice Lake servers. With a representative 120-VF configuration, DMMU improves NIC throughput by 65% and reduces IOTLB misses by 38 × compared to the baseline IOMMU. DMMU also enables reliable memory overcommitment with device passthrough, achieving near-baseline Redis performance at a 1.35 × overcommitment ratio.