ACALSim: A Scalable Parallel Simulation Framework for High-Performance System Design Space Exploration

Architectural simulation has become the critical bottleneck limiting design space exploration for high-performance computing systems. The growing complexity of modern GPUs and AI accelerators—with hundreds to thousands of tightly-coupled components—demands simulation frameworks that deliver efficient parallelism and scalable single-node execution. Existing frameworks such as SST, gem5, and GPGPU-Sim fail to meet these requirements: SST focuses on multi-node MPI scalability but struggles with intra-node scaling; GPGPU-Sim remains largely single-threaded. Critically, these frameworks provide fixed threading models with no mechanism for users to optimize simulation performance for their specific workload patterns. We introduce ACALSim, a scalable parallel simulation framework designed to accelerate design space exploration for high-performance systems. As a framework contribution, ACALSim provides infrastructure and APIs for building high-performance simulators—timing model accuracy is the responsibility of simulator developers, not the framework. ACALSim's key innovation is a pluggable thread management architecture that enables developers to implement custom scheduling strategies optimized for specific simulation patterns—a capability absent in existing frameworks. This is complemented by (1) event-driven execution with fast-forward to eliminate idle cycle overhead, (2) a shared-memory data model enabling zero-copy communication, and (3) a two-phase parallel execution model for deterministic thread scaling. We demonstrate ACALSim's effectiveness through HPCSim, a GPU simulator targeting A100-class architectures. Direct comparison with an SST implementation—using identical shared timing cores to isolate framework overhead—shows ACALSim achieves over 14× speedup with 41% lower memory footprint, while hardware validation confirms 0.72--1.22× cycle count correlation with A100 measurements. While SST fails to complete 256+ thread block workloads within practical time limits, ACALSim simulates full LLaMA transformer layers (single block) in 17.7 minutes for LLaMA-7B and 30.4 minutes for LLaMA-13B—enabling practical design space exploration that SST cannot achieve.