Evaluation of Dialogue Systems Functioning Efficiency based on Using Fuzzy Inference with Neural Network Setting

The results of the study aimed at improving the process of evaluating the functioning efficiency of dialogue systems based on the development of models of fuzzy inference and neural network learning are presented. Developing tools to assess how virtual assistants actually perform on question-answer tasks is a practical concern in Software Engineering - one the field has not yet solved cleanly. As values for evaluating the functioning efficiency of dialogue systems, it is proposed to use particular indicators that characterize the accuracy, concision and fullness of answers to the questions asked. The resulting performance evaluation is determined by the value of the generalized indicator, calculated taking into account the values of particular indicators. An algorithm for calculating a generalized indicator based on the use of fuzzy inference has been developed. The values of parameters of membership functions and individual conclusions of fuzzy rules necessary for its implementation are proposed to be calculated on the basis of a neural network learning algorithm. The decision to complete the neural network tuning of the fuzzy inference parameters is made based on the calculation and analysis of the current learning error. The results of experimental studies on evaluating the functioning efficiency of dialogue systems based on the proposed algorithms are presented. Across 1,012 conducted experiments, the composite functioning efficiency indicator ranged from 0.395 to 0.912, and the ANFIS training converged after 12 epochs to a minimum error of 0.001996.