Fine-tuning and retrieval augmentation for crystallography and cryo-EM question answering using a community-derived corpus

Structural biology mailing lists such as CCP4, CCP-EM, Phenix and COOT contain decades of expert answers to practical problems, but an answer is difficult to find even when it has been given before. A question-and-answer corpus was built from 103,600 messages across the four mailing lists, and threads were classified as questions. The answers were confirmed by the asker’s reply or by the agreement of several participants, identified by three LLM models. This gave 4,320 curated question-and-answer pairs, which were used for fine-tuning and retrieval augmentation for six LLM models, two sizes each from the Mistral, Llama and Qwen families. The LLM models were evaluated under eight conditions: base and fine-tuned, each without retrieval and with retrieval from the software documentation, from the mailing-list corpus, and from both. The base models with mailing-list retrieval were preferred by the LLM judges over the base models alone, the fine-tuned models, and even the base models with retrieval from the software documentation. The data sets and the fine-tuned models are publicly available.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-19
DOI
https://doi.org/10.21203/rs.3.rs-11741379/v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Fine-tuning and retrieval augmentation for crystallography and cryo-EM question answering using a community-derived corpus

Emad Alharbi
Research Square
Topic Modeling
preprint

Fine-tuning and retrieval augmentation for crystallography and cryo-EM question answering using a community-derived corpus

Emad Alharbi
preprint en

Abstract

Structural biology mailing lists such as CCP4, CCP-EM, Phenix and COOT contain decades of expert answers to practical problems, but an answer is difficult to find even when it has been given before. A question-and-answer corpus was built from 103,600 messages across the four mailing lists, and threads were classified as questions. The answers were confirmed by the asker’s reply or by the agreement of several participants, identified by three LLM models. This gave 4,320 curated question-and-answer pairs, which were used for fine-tuning and retrieval augmentation for six LLM models, two sizes each from the Mistral, Llama and Qwen families. The LLM models were evaluated under eight conditions: base and fine-tuned, each without retrieval and with retrieval from the software documentation, from the mailing-list corpus, and from both. The base models with mailing-list retrieval were preferred by the LLM judges over the base models alone, the fine-tuned models, and even the base models with retrieval from the software documentation. The data sets and the fine-tuned models are publicly available.

Research Square
University of Tabuk (SA)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon