SplitSim: Towards Practical Large-Scale Full-System Simulation for Systems Research

When physical testbeds are out of reach for evaluating a networked system, we frequently turn to simulation. In today's datacenter networks, bottlenecks are often not at the network protocol level, but instead in end-host software or hardware components. Here, current protocol-level simulations are an inadequate means of evaluation. Detailed end-to-end simulations covering these components on the other hand, simply cannot achieve the required scale with feasible simulation performance and computational resources. It is fundamentally difficult to simultaneously achieve high simulation fidelity, low simulation time, and minimal resource consumption-especially for large-scale systems. With limited time and resource budgets, users must find the right compromise across these dimensions. Unfortunately, existing simulation frameworks do not offer flexible trade-offs among these aspects and are unable to make effective use of the time and resources users can afford. In this paper, we address this with SplitSim, a simulation framework for practical end-to-end evaluation for large-scale network and distributed systems. SplitSim provides the user abstractions to specify the system to be evaluated, and easily & flexibly instantiate different simulations for the same system to navigate these trade-offs. Based on these abstractions, SplitSim allows users to flexibly trade off simulation fidelity, time, and resource usage by mixing simulation models with different fidelity and controlled parallelization. Finally, SplitSim includes a profiler that identifies simulation performance bottlenecks and helps users make informed decisions about parallelization, resource allocation, and simulation detail. With these capabilities, we demonstrate that SplitSim can simulate a 1200-node datacenter network using 24 cores running end-to-end applications in 175 min for 20 s of system time.