Lifelong Memory Architectures for Large Language Model Based Generative AI Systems: A Survey and Comparative Analysis

Large Language Models (LLMs) form the basis of multiple generative applications today, but these systems are mostly stateless which means that information keeps outside the context window disappears when the session is closed. Such limitations complicate personalization, make it difficult to maintain consistency over long interactions, and prevent the AI system from learning from its experience. Lifelong memory architectures address this problem by providing the LLM with an external, permanent memory that also includes means of information writing, retrieval, updating, consolidating, and forgetting without the need to retrain the model itself. This paper presents an overview of the developments in this area starting from early external-memory enhancement and reflective generative agents, and continuing with memory operating system, graph-shaped memory, and reinforcement learning of agent memory. We summarize our results, suggesting an operation-centered classification of such systems and compare such twelve representations with regard to scalability, retrieval efficiency, personalization, computational cost, and adaptability, discussing necessary technologies such as embeddings, approximate nearest neighbor index, knowledge graph, and retrieval-augmented generation.