Why Retrieval is not Enough: Structured Memory Scheduling for Large Language Model Reasoning

Retrieval Augmented Generation (RAG) has emerged as a mainstream paradigm for mitigating hallucinations in large language models and supporting knowledge-intensive complex reasoning tasks. However, existing retrieval-as-context methods typically employ either one-shot retrieval or blind iteration strategies, lacking global planning and dynamic scheduling for heterogeneous knowledge. This often introduces substantial redundant information and disrupts the reasoning process. To address these issues, we propose structured memory scheduling retrieval augmented generation (SMS-RAG) framework. SMS-RAG explicitly models external knowledge as compound memories with semantic, structural, and episodic features. It introduces a structured scheduling mechanism to explicitly model and control the activation timing, sequence, and dependency relationships among different knowledge. This enables knowledge to participate in reasoning in a structured manner, rather than merely serving as passive contextual input. It consists of three modules: MemPlanner determines which memory modules to activate by generating a task-aware MemoryPlan; MemScheduler decides in what order to activate them by constructing a dependency-aware graph; and MemRouter executes how to activate by selectively routing across semantic, structural, and episodic memory layers to extract critical information for reasoning. Extensive experiments on three multi-hop QA benchmarks and a domain-specific legal QA dataset demonstrate that SMS-RAG significantly outperforms existing state-of-the-art methods in reasoning accuracy, factual coverage, and scheduling efficiency.