Comparing the accuracy and efficiency of a single machine learning agent with multiple coordinating agents in a chat environment

This paper aims to design a chatbot application to help people find an appropriate restaurant based on users' preferences and transportation modes. It also aims to explore whether a machine learning-based chat agent is capable of accomplishing all the tasks precisely, or whether it could be optimized by collaborating with other simple agents. This paper establishes a chatbot environment along with a series of agents (machine learning agent, grammar checker agent, spelling checker agent, restaurant searcher), designed to assist users in finding a restaurant mainly based on what transportation they use and if a restaurant is found, the chatbot could display the route to this place in the prompt also with map details by employing the Google map API. The machine learning agent acting as the brain of this chatbot, could not only be able to evoke other agents corresponding to users’ input but also could directly store what kinds of transportation are taken by users and pass to Google map API to search appropriate restaurants instead of fully relying on a restaurant searcher agent (rule-based agent). To examine whether multiple agents coordinate with each other could outperform a single agent. A contrast experiment is conducted to conclude the result, which involves 15 participants who run through this program under two different setups (with and without spelling and grammar checkers) and each of them would give a query to this chatbot but this query would be dealt with and without spelling and grammar checker respectively. The result would be quantitatively evaluated by two metrics: Success Rate: successfully finding a restaurant and showing the map; the Total of Queries. Results suggest that pre-processing input with spelling and grammar checkers significantly improves the chatbot's performance, achieving an 80% success rate (12 out of 15 queries). In contrast, when input is not pre-processed, the success rate drops to 46.66% (7 out of 15 queries). This indicates that the involvement of spelling and grammar checker agents enhances the accuracy and efficiency of restaurant searches, making the collaborative multi-agent setup more effective. Moreover, it could demonstrate that this chatbot environment is well constructed since it could successfully provide suitable restaurants given users’ requirements.