What’s the data say? An LLM-based system for interrogating experimental data

Large Language Models (LLMs) have demonstrated remarkable capabilities across a variety of domains. However, issues such as hallucinations, implicit extrapolations, and inappropriate interpretations currently limit their wide adoption in scientific research. Retrieval-augmented Generation (RAG) has emerged as a promising technique to enhance the accuracy and reliability of LLM outputs. RAG integrates external information with LLM-generated content, ensuring more dependable results and enabling the inclusion of proprietary or domain-specific knowledge unavailable during the initial LLM training. In this work, we present an extensible LLM RAG architecture designed to answer complex scientific questions that can reference user-provided experimental data. The system uses domain-specific knowledge (RAG<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dom</inf>) to respond to general inquiries, supporting answers with citations from authoritative sources. These responses can be recursively integrated to provide user directed context for experiments (RAG<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">exp</inf>), allowing subsequent questions to utilize both general and experiment-specific information. Our system can be easily extended with user-defined domain-specific modules for functionally interrogating quantitative experimental results (RAG<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fun</inf>). We demonstrate its application in analyzing, summarizing, interpreting, and assisting with synthesizing results from a gene expression comparison of two phenotypes. The RAG<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dom</inf> component encompasses biological pathways, genes, and disease-related knowledge. The RAG<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">exp</inf> component is derived from user queries of differentially expressed gene (DEG) and pathway enrichment analysis (PEA) results. Additionally, RAG<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fun</inf> modules are designed to interface with DEG and PEA data, enabling quantitative experimental results to be integrated seamlessly into user questions. Unlike traditional LLM outputs, our system’s results are free of hallucinations and include paragraph-level citations to the supporting literature. We compared the quality of our system’s output with those generated by LLM-only models such as GPT-3, GPT-4, Llama2, PaLM2, and BioGPT, and found our approach to be superior.