Scalar Vector Runahead: Removing the Shackles of Indirect Memory Chains on In-Order Cores

Modern processors often face the memory wall as a significant bottleneck, a problem that becomes particularly severe when using stall-on-use in-order cores. Despite this limitation, there is growing demand for energy-efficient in-order cores due to privacy and sustainability concerns. Scalar Vector Runahead (SVR)1 provides an elegant solution by extracting high memory-level parallelism through piggybacking on existing instructions executed on the processor that lead to future irregular memory accesses. SVR speculatively executes multiple transient, independent, parallel instances of memory accesses and their instruction chains, by initiating memory accesses from many different values of a predicted induction variable. This approach moves mutually independent memory accesses next to each other to hide dependent stalls. With a hardware overhead of only 2 KiB and without the need for hardware vector extensions, SVR delivers 3.2× higher performance than a baseline 3-wide in-order core inspired by an Arm Cortex A510, and 1.3× higher performance than a full out-of-order core, while halving energy consumption.