Quantifying the quality of responses generated by augmented large language models

This paper discusses a methodology for comparing the quality of responses generated by large language models. It is proposed to select six generative models and three words embedding for analysis. The generation quality of the models is examined using three knowledge base variations: a structured document, an unstructured document, and both. The values of quantitative metrics used to evaluate natural language processing models are presented as a means to examine the responses. This approach allows the most appropriate architecture to be selected for analysing documents format. A comparative analysis of language models enhanced with retrieval augmented generation technology has been conducted. As a result, conclusions about the quality of the models are drawn and the best models are selected.