Digital Resources to Long-Term Agent Memory: Evaluating Memory Formation and Retrieval Across Heterogeneous Personal Knowledge

The ability to retain and retrieve information from heterogeneous digital resources is an important requirement for long-term memory in intelligent agents. This paper evaluates different memory representation and retrieval approaches for organizing personal knowledge collected from multiple digital sources. The evaluated dataset contains 100 resources across five categories: webpages, video transcripts, GitHub repositories, research papers, and personal notes. Of these, 96 resources were successfully processed. A question set containing 150 questions was used, with 30 questions allocated for development and 120 for testing. Five approaches were compared: BM25 retrieval, dense retrieval-augmented generation (RAG), summary-based memory, structured-fact memory, and graph-linked memory. The evaluation considered retrieval effectiveness, answer accuracy, evidence support, fact extraction quality, provenance preservation, and storage and retrieval efficiency. Among the evaluated approaches, graph-linked memory achieved the highest retrieval performance, with a Recall@5 of 0.83, MRR of 0.70, and nDCG@5 of 0.76. It also achieved the highest answer accuracy at 77%. Structured-fact memory achieved the highest fact extraction F1-score of 0.90 and provenance score of 0.98, while requiring less storage and providing lower median retrieval latency than graph-linked memory. These findings illustrate the trade-offs between retrieval effectiveness, memory representation quality, and computational efficiency when building long-term memory from heterogeneous personal knowledge.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-15
DOI
https://doi.org/10.5281/zenodo.24241304
Primary Topic
Information Retrieval and Search Behavior
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Digital Resources to Long-Term Agent Memory: Evaluating Memory Formation and Retrieval Across Heterogeneous Personal Knowledge

vansh, kishore sabavath, devansh
Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
article

Digital Resources to Long-Term Agent Memory: Evaluating Memory Formation and Retrieval Across Heterogeneous Personal Knowledge

vansh, kishore sabavath, devansh
article en

Abstract

The ability to retain and retrieve information from heterogeneous digital resources is an important requirement for long-term memory in intelligent agents. This paper evaluates different memory representation and retrieval approaches for organizing personal knowledge collected from multiple digital sources. The evaluated dataset contains 100 resources across five categories: webpages, video transcripts, GitHub repositories, research papers, and personal notes. Of these, 96 resources were successfully processed. A question set containing 150 questions was used, with 30 questions allocated for development and 120 for testing. Five approaches were compared: BM25 retrieval, dense retrieval-augmented generation (RAG), summary-based memory, structured-fact memory, and graph-linked memory. The evaluation considered retrieval effectiveness, answer accuracy, evidence support, fact extraction quality, provenance preservation, and storage and retrieval efficiency. Among the evaluated approaches, graph-linked memory achieved the highest retrieval performance, with a Recall@5 of 0.83, MRR of 0.70, and nDCG@5 of 0.76. It also achieved the highest answer accuracy at 77%. Structured-fact memory achieved the highest fact extraction F1-score of 0.90 and provenance score of 0.98, while requiring less storage and providing lower median retrieval latency than graph-linked memory. These findings illustrate the trade-offs between retrieval effectiveness, memory representation quality, and computational efficiency when building long-term memory from heterogeneous personal knowledge.

Zenodo (CERN European Organization for Nuclear Research)
Information Retrieval and Search Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Digital Resources to Long-Term Agent Memory: Evaluating Memory Formation and Retrieval Across Heterogeneous Personal Knowledge — vansh, kishore sabavath, et al. · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS