Evaluation of a User Interface Extension Integrating anLLM-Based AI Assistant into an Interactive Visual EquationEditor for Solving High School Mathematics Problems

This paper presents an evaluation of AI-assisted human–computer interaction for mathematicalproblem solving within a multimodal mathematical editing environment. Theproposed architecture integrates a visual equation editor, voice-based interaction, RESTcommunication services, and externally hosted generative AI large language models (LLMs)to support the creation, interpretation, and solution of mathematical expressions. The studywas conducted using the Equation Wizard environment, which provides multimodal mathematicalcontent editing based on both proprietary and standard formula representations,including MathML and LATEX. A conversational AI interaction layer enables users to communicatewith selected LLMs using natural language voice commands. The main objectiveof the study was to determine whether contemporary LLM-based services can reliablysupport mathematical problem solving within an AI-enhanced equation editing environment.To address this objective, seven contemporary GenAI LLMs were evaluated usinga benchmark consisting of representative high school mathematics problems coveringalgebra, limits, trigonometry, logarithms, inequalities, and parameterized expressions. Theevaluation focused on mathematical correctness, output interpretability and visualizationquality, response latency, and compliance with mathematical encoding formats within thecomplete interaction workflow. The study also compares representative model familieswith respect to correctness, syntactic compliance, and responsiveness within the completeinteraction workflow. Unlike conventional LLM benchmarks that assess models in isolation,this work evaluates end-to-end AI-assisted mathematical interaction involving the editor,communication infrastructure, and language models. The study demonstrates a practicalapproach to assessing the usefulness of multimodal AI-enhanced mathematical problemsolving in realistic usage scenarios. The results show that, within the evaluated interactionworkflow, current LLMs achieve high levels of mathematical problem-solving performance,although substantial differences were observed in response latency and output-formatcompliance. An important finding is the discrepancy between the models’ strong semanticunderstanding of custom-encoded mathematical input and their weaker ability to generatesyntactically compliant encoded outputs, highlighting a key challenge in the integration ofLLMs with structured mathematical software environments.