BHT-RAG: Birch-based Hierarchical Tree RAG in Mass Spectrometry

This manuscript presents a Birch-based hierarchical tree retrieval-augmented generation method (BHT-RAG) to mitigate the limitations of information retrieval insufficiency and retrieval quality in the RAG (Retrieval-Augmented Generation) approach within mass spectrometry. Specifically, the documents were initially divided into chunks, and each chunk was then vectorized, clustered, and summarized utilizing Birch and a language generation model. This hierarchical clustering culminated in a recursive, tree-structured search tree. To evaluate RAG in the mass spectrometry domain, MSeval, a publicly available RAG evaluation dataset specific to medical mass spectrometry, was further constructed and released. MSeval involves documents for retrieval, question-answer pairs, multiple-choice questions, and fill-in-the-blank questions. In the experimental section, various combinations of clustering methods and generators were initially assessed using the mass spectrometry RAG dataset to gauge their generalization across different tasks, and the optimal generator and clustering method were selected. Subsequently, under the same generator conditions, four other RAG retrieval methods were compared. The experimental results indicate that in the mass spectrometry dataset, BHT-RAG achieves BLEU-4, ROUGE-L, and METEOR scores of 16.7366, 31.1675, and 40.8344 on question-answer tasks; precision, recall, and F1 scores of 90.19%, 89.67%, and 89.93% on multiple-choice tasks; and a maximum accuracy of 80.20% on fill-inthe-blank tasks. BHT-RAG demonstrates superior model generalization across all three tasks compared to other models, showcasing certain effectiveness. Additionally, supplementary ablation experiments, both with and without the clustering module, were conducted to validate its impact. Finally, to maintain experimental integrity, tests were conducted on two other publicly available RAG datasets, Wikieval and L-eval, yielding results that corroborate the effectiveness of the proposed approach.