Split gRPC: An Isolation Architecture for RPC Software Stacks

Remote procedure calls are a major contributor to performance variance in distributed systems due to lack of isolation and contention on shared resources. We propose a novel split architecture for a popular RPC framework, gRPC. The design partitions RPC applications into two communicating components: one dedicated to user-implemented business logic, and one dedicated to RPC infrastructure processing. The infrastructure process can be run on a dedicated core or on a smart NIC (e.g., IPU or DPU), providing effective physical isolation and predictable performance. An initial evaluation shows that the split architecture adds modest overhead for average case latency, but allows for lower latency and and higher throughput under host CPU load.