TAP: Transformers Driven Adaptive Prefetching for Hybrid Memory Systems

In the rapidly evolving landscape of hybrid memory systems, which integrate different types of memory, optimizing cache management is vital for achieving high performance and efficiency. Traditional prefetching techniques often struggle to adapt to modern systems’ dynamic and complex I/O access patterns. This paper proposes a novel machine learning-based cache replacement framework tailored for hybrid memory architectures. Our framework leverages advanced transformerbased sequence-to-sequence models to analyze historical access patterns and make data-driven cache management decisions. By continuously learning from past access behavior and adapting to real-time workload changes, our system enhances cache hit rates, reduces latency, and optimizes storage performance. Our Transformer-based adaptive prefetching approach addresses the problem of predicting within large address spaces and shows significant improvements over traditional (Stride) and machine learning (ML) based (Markov Chains, Long Short Term Memory) prefetchers, achieving hit ratios of 0.71, 0.91, and 0.47 for mixed, streaming, and random access workloads. It yields 16-64% improvements across various traces and settings by capturing complex contextual relationships and adapting to dynamic access patterns, enhancing hybrid memory systems and enabling efficient, scalable storage management in modern computing environments.