RELOAF: A Reinforcement Learning-based Online Adaptive Filtering Framework for Data Prefetching

Data prefetching techniques have been widely adopted to resolve the bottleneck of access latency between processors and memory. However, existing prefetchers, while effective in hiding memory access latency, often aggressively prefetch large amounts of overpredictions into caches in advance. This behavior can result in cache pollution, bandwidth waste, and even degrade overall system performance. To resolve this issue, numerous filters have been integrated into prefetchers to eliminate overpredictions during prefetching. Nevertheless, existing mainstream filters in prefetchers typically suffer from two critical limitations. First, they lack the awareness to system performance , neglecting the overall impact of prefetch decisions on the system. Second, they exhibit insufficient adaptivity to program behaviors , making them incapable of dynamically adjusting to changes in different programs or execution stages. To overcome these limitations, we propose RELOAF, a RE inforcement L earning-based O nline A daptive F iltering framework that can be integrated into different prefetchers. Specifically, RELOAF comprises two key components: a system information feedback component and an adaptive reinforcement learning (RL) decision component. In system information feedback component, system-level feedback information, such as prefetch accuracy, coverage, and IPC, is introduced as a decision criterion to evaluate the validity of prefetching, thereby resolving the first limitation. In the adaptive RL decision component, RELOAF dynamically adjusts prefetching based on the current program state using reinforcement learning, thus handling the second limitation. Experiments validate the effectiveness of the proposed approach. Results on benchmark workloads show that augmenting state-of-the-art prefetchers with RELOAF improves their average prefetch accuracy by 3.65%–6.49% and IPC by 0.81%–1.83%, compared to the same prefetchers operating without RELOAF.