Enhanced Prefetching via Dynamic Multi-Step SARSA based Reinforcement Learning

Cache prefetchers are essential for system performance, and reinforcement learning (RL) offers a lightweight alternative to heavy machine learning techniques for modern prefetchers. While previous work used 1-step SARSA-based RL for cache prefetching, it can struggle with complex memory access patterns. This study proposes using n-step SARSA instead of 1-step to capture broader contextual learning. We incorporate a dynamic property to reduce the use of outdated Q-values, guiding the RL agent to improve prefetching effectiveness. Our approach achieves approximately 12% better performance than non-RLbased prefetchers in multi-core setups and also outperforms previous RL-based prefetcher.