User-driven development and evaluation of an agentic framework for analysis of large pathway diagrams

Abstract As biomedical knowledge keeps growing, resources storing available information multiply and grow in size and complexity. Such resources can be in the format of molecular interaction maps, which represent cellular and molecular processes under normal or pathological conditions. However, these maps can be complex and hard to navigate, especially to novice users. Large Language Models (LLMs), particularly in the form of agentic frameworks, have emerged as a promising technology to support this exploration. In this article, we describe a user-driven process of prototyping, development, and user testing of Llemy, an LLM-based system for exploring these molecular interaction maps. By involving domain experts from the very first prototyping in the form of a hackathon and collecting both fine-grained and general feedback on more refined versions, we were able to evaluate the perceived utility and quality of the developed system, in particular for summarising maps and pathways, as well as prioritise the development of future features. We recommend continued user-driven development and benchmarking to keep the community engaged. This will also facilitate the transition towards open-weight LLMs to support the needs of the open research environment in an ever-changing technology landscape.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-03-12
DOI
https://doi.org/10.64898/2026.03.10.710813
Primary Topic
Biomedical Text Mining and Ontologies
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

User-driven development and evaluation of an agentic framework for analysis of large pathway diagrams

L. Geris, M. Kutmon, Matthias König, Eliška Kuchovská et al.
bioRxiv (Cold Spring Harbor Laboratory)
Biomedical Text Mining and Ontologies
article

User-driven development and evaluation of an agentic framework for analysis of large pathway diagrams

L. Geris, M. Kutmon, Matthias König, Eliška Kuchovská, Marek Ostaszewski, L.C.M. Ladeira, Lorenzo Veschini, Tim Hofer, Ahmed Abdelmonem Hemedan, Alexander Mazein, Marie Corradi, Marc Teunis, Rupert W. Overall, Anna Niarakis, Julen Sanz-Serrano, Reinhard Schneider, Adrien Rougny, Arthur C. Silva, Noriko Hiroi, Ivo Djidrovski, Ahmad Vaez, Oxana Lopata, Egon L. Wilighagen, A. J. Verhoeven, Matti van Welzen, Pierre Klemmer, Lisa Matthews, Irina Balaur, Marc Gillespie, Venkata Satagopam, Charles Auffray, Laurence Calzone, Falk Schreiber, Bernard Staumont
article en

Abstract

Abstract As biomedical knowledge keeps growing, resources storing available information multiply and grow in size and complexity. Such resources can be in the format of molecular interaction maps, which represent cellular and molecular processes under normal or pathological conditions. However, these maps can be complex and hard to navigate, especially to novice users. Large Language Models (LLMs), particularly in the form of agentic frameworks, have emerged as a promising technology to support this exploration. In this article, we describe a user-driven process of prototyping, development, and user testing of Llemy, an LLM-based system for exploring these molecular interaction maps. By involving domain experts from the very first prototyping in the form of a hackathon and collecting both fine-grained and general feedback on more refined versions, we were able to evaluate the perceived utility and quality of the developed system, in particular for summarising maps and pathways, as well as prioritise the development of future features. We recommend continued user-driven development and benchmarking to keep the community engaged. This will also facilitate the transition towards open-weight LLMs to support the needs of the open research environment in an ever-changing technology landscape.

bioRxiv (Cold Spring Harbor Laboratory)
University of Stuttgart (DE), Isfahan University of Medical Sciences (IR), Centre National de la Recherche Scientifique (FR), Norwegian Institute of Public Health (NO), University Medical Center Groningen (NL), Vrije Universiteit Brussel (BE), St. John's University (US), Inserm (FR), University of Liège (BE), University of Applied Sciences Utrecht (NL), Université Fédérale de Toulouse Midi-Pyrénées (FR), Kanagawa Institute of Technology (JP), University of Luxembourg (LU), University of Konstanz (DE), Cal Poly Humboldt (US), Utrecht University (NL), Université Paris Sciences et Lettres (FR), Maastricht University Medical Centre (NL), Institute for Systems Biology (US), Maastricht University (NL), St. John's University (TW), InSysBio (Russia) (RU), Leibniz Institute of Environmental Medicine (DE), Luxembourg Institute of Health (LU), Biomechanics Institute of Valencia (ES), Indiana University Bloomington (US), European Institute for Systems Biology and Medicine, University of Rostock (DE), New York University (US), University of Birmingham (GB)
European Commission, Nederlandse Organisatie voor Wetenschappelijk Onderzoek, HORIZON EUROPE Framework Programme
Openalex Percentile: Top 25%
Biomedical Text Mining and Ontologies
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.