Large language models and speech genre systematicity

The paper examines a large language model (LLM) to recognize speech genres. Although artificial neural networks are effectively utilized in many important fields, they, however, have a serious drawback. The mechanism of their functioning is hidden from researchers; therefore, the results of their use do not get explanation. The purpose of the study is to reveal the basic mechanisms of functioning of the linguistic model LLM (Transformer) and thereby ensure the interpretability of the data it provides. The research is based on two genres of academic text: “Description of a new scientific phenomenon” and “Explication of a scientific concept.” We verified a hypothesis according to which the LLM feature set is based on the speech systematicity of the recognized genres. It is also shown that since genre-speech systematicity is determined by extralinguistic factors, primarily the characteristics of human consciousness, its manifestations, reflected in the hidden state of the LLM, can be used to model cognitive processes embodied in speech. We also analyze existing approaches to the interpretation of LLMs and describe the applied method to do it. The paper provides the following linguistic interpretation of LLM training and fine-tuning: preliminary training on large text corpora allows a model to display language resources (a system of linguistic units and general principles of their use) relatively completely, while fine-tuning on samples of a certain genre-speech organization restructures the linguistic systematicity into speech systematicity. During the experiments we decoded the hidden state of the LLM and accurately reproduced the composition and frequency of lexis from the training dataset. The classification score for each of the considered genres by the LLM is F1 0.99, we believe this is because of their speech consistency.