Design of a data prefetcher using hierarchical memory sharing prediction for multi-die architectures

This paper presents a novel hierarchical memory sharing predictor (HMSP), consisting of two independent memory sharing predictors, namely MSP-C and MSP-D, designed for data prefetching in multi-die architectures. HMSP addresses the performance bottleneck of shared data accesses, which arise not only from the data transmission delays but also from the execution overhead of coherence protocol in multi-die systems, by proactively forwarding data blocks from producers to consumers. Unlike previous approaches which generate prefetch addresses at the consumer side to speculatively pull the shared data blocks from their producers, HMSP initiates forwarding a data block from its producer to its consumers at the earliest time, i.e., as soon as the write process of a data block is completed at the producer side, to effectively hide the abovementioned delays and overhead. MSP-C and MSP-D cooperate to realize this forwarding mechanism, even though they reside in different locations: MSP-C at the CPU die and MSP-D at the I/O die. Across the evaluated benchmarks, MSP-C and MSP-D achieved average prediction coverages of 81.3% and 83.3%, respectively. The central idea of HMSP is to dynamically learn correlations between writers and their subsequent readers during execution. When a modified block is speculatively downgraded, HMSP uses the received address along with the identity of the downgrading core or die to look up for the correlation in prediction tables and forwards the data to the predicted sharers accordingly; thereby hiding the latency of coherence misses undertaken by the reading cores. Experimental results demonstrate that HMSP improves performance by an average of 17.75%, with up to 43% improvement in the best case.