LPGSim: A Lightweight Parallel GPU Simulator Maximizing Speed with Trustworthy Simulation

The demands of large-scale workloads have driven the evolution of GPUs, placing them in the mainstream scope of computing architectures. To design an effective GPU for large-scale workloads, a trustworthy simulator is required to evaluate performance and explore the design space. Additionally, GPU simulators must be fast enough to evaluate architectural modifications for large-scale workloads within a reasonable time. However, existing GPU simulators suffer from long execution times due to detailed component simulation, limiting their utility for evaluating the effects of architectural modifications on large-scale workloads. It is necessary to improve the performance of a GPU simulator such that quick architecture exploration and evaluation for large-scale workloads are available, at the expense of accuracy. This paper presents LPGSim, a trace-driven and cycle-level GPU simulator. LPGSim aims to provide fast and accurate GPU simulation. To this end, LPGSim first eliminates instruction metadata that has minimal impact on simulation accuracy. This approach enables coalescing compute instructions while preserving the dependencies to the preceding memory instruction. As a consequence, the core pipeline is simplified, improving simulation speed at a marginal cost in accuracy. Next, LPGSim parallelizes GPU simulation. LPGSim partitions a GPU architecture into three parallelizable subsystems and introduces local-clock-based parallelization to reduce the overhead of global synchronization and sequential execution paths in existing methods. LPGSim further employs parallelization methods to achieve scalability on NUMA systems with an acceptable trade-off in accuracy. Our evaluation shows a modest decrease in simulation accuracy in return for a substantial improvement in simulation speed. In terms of accuracy, LPGSim achieves 21.4%, 23.3%, and 22.7% errors across three different GPU architectures. In terms of speed, the single-threaded LPGSim simulation yields a 9.97x speedup over the state-of-the-art GPU simulator. Parallelization achieves a 19.8x speedup on a two-socket 56-thread system, for a total 197.4x speedup over the state-of-the-art simulator. Parallelization incurs an additional accuracy loss, but our experiments indicate that simulation is still reliable for large-scale workloads.