Enhanced System-Level Coherence for Heterogeneous Unified Memory Architectures

Heterogeneous Unified Memory Architectures (HUMA) provide a unified memory space for on-die CPUs, GPUs, and other hardware accelerators. Such architectures improve performance and energy efficiency by obviating explicit data transfers between processors. An important feature of such architectures is Heterogeneous System Coherence (HSC) which simplifies the programming model by reducing the explicit synchronizations otherwise expected of the programmers of such systems. However, due to differences in the memory models and bandwidth requirements of CPUs and GPUs, hardware implementation of coherence for such systems is often complex and comes at high power, performance, and area trade-offs.This paper optimizes the existing heterogeneous coherence mechanism in early AMD Accelerated Processing Units, approximately modeled in the gem5 simulator. It introduces precise sharing information in the system-level directory, which monitors both CPU and GPU cache lines, and implements a new write-back shared last-level cache (LLC). The original implementation consisted of a stateless system-level directory and a write-through LLC. Our evaluation results with a set of collaborative heterogeneous benchmarks reveal, on average, a 14.4% performance improvement and 80.8% and 50.4% reduced probing traffic and main-memory interactions, respectively. Through optimizations and adaptation of the evaluated benchmarks, this work aims to reduce the barriers to entry into HSC research.