MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation
Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 32K or 128K tokens), it is computationally expensive and can still be insufficient for many applications. Retrieval-Augmented Generation (RAG) is considered a promising strategy to address this problem. However, conventional RAG methods face inherent limitations because of two underlying requirements: 1) explicitly stated queries, and 2) well-structured knowledge. These conditions, however, do not hold in general long-context processing tasks.
Authors
- Hongjin Qian (ORCID: https://orcid.org/0000-0003-4011-5673)
- Zhicheng Dou (ORCID: https://orcid.org/0000-0002-9781-948X)
- Tiejun Huang (ORCID: https://orcid.org/0000-0002-4234-6099)
- Kelong Mao (ORCID: https://orcid.org/0000-0002-5648-568X)
- Peitian Zhang (ORCID: https://orcid.org/0009-0007-1926-7433)
- Defu Lian (ORCID: https://orcid.org/0000-0002-3507-9607)
- Zheng Liu (ORCID: https://orcid.org/0000-0001-7765-8466)
Institutions
- University of Science and Technology of China (CN)
- Hong Kong Polytechnic University (HK)
- Peking University (CN)
- Beijing Academy of Artificial Intelligence (CN)
- Renmin University of China (CN)
Publication Details
- Published
- 2025-04-22
- DOI
- https://doi.org/10.1145/3696410.3714805
- Citations
- 7
- Primary Topic
- Advanced Image and Video Retrieval Techniques
- Type
- article
- Field-Weighted Citation Impact
- 8.93
Funders
- National Natural Science Foundation of China
- Ministry of Education, India