EXPERT-ORIENTED METHODS FOR EVALUATING THE QUALITY OF TEXTUAL GENERATION OF LARGE LANGUAGE MODELS

This paper is devoted to a systematic analysis of approaches in which human; expert markup remains the key source of reliable judgments. The aim of the paper is to identify the strengths and weaknesses of manual evaluation compared to traditional automatic metrics (BLEU, ROUGE, BERTScore, etc.) and stateoftheart LLM-as-a-Judge schemes. The analysis is based on 2023-2025 research covering both foreign (OpenAI, Anthropic, DeepMind) and domestic instructional benchmarking initiatives. Manual evaluation is confirmed as the "gold standard" due to the consideration of experts' deep sense and common sense but demonstrates low scalability and high cost. Automated ngram metrics are often at odds with expert judgment on open tasks. The novel LLM-as-a-Judge approach provides high overlap with human decisions but is subject to its own systematic biases and requires calibration based on expert markup. The scientific novelty of the paper consists in a comprehensive classification of expertbased techniques and formulation of practical recommendations for their integration with LL-Massessors. The practical significance lies in the possibility of reducing annotation costs while maintaining the accuracy of the estimation in the development and operation of LLM systems.