Chat with MES: LLM-driven user interface for manipulating garment manufacturing system through natural language

This paper presents Chat with MES (CWM), an AI agent system, which integrates LLMs into the Manufacturing Execution System (MES), serving as the “ears, mouth, and the brain”. This system promotes a paradigm shift in MES interactions from Graphical User Interface (GUI) to natural language interface”, offering a more natural and efficient way for workers to manipulate the manufacturing system. Compared with the traditional GUI, both the maintenance costs for developers and the learning costs and the complexity of use for workers are significantly reduced. This paper also contributes two technical improvements to address the challenges of using LLM-Agent in serious manufacturing scenarios. The first one is Request Rewriting, designed to rephrase or automatically follow up on non-standardized and ambiguous requests from users. The second innovation is the Multi-Step Dynamic Operations Generation, which is a pre-execution planning technique similar to Chain-of-Thought (COT), used to enhance the success rate of handling complex tasks involving multiple operations. A case study conducted on a simulated garment MES with 55 manually designed requests demonstrates the high execution accuracy of CWM (80%) and the improvement achieved through query rewriting (9.1%) and Multi-Step Dynamic operations generation (18.2%). The source code of CWM, along with the simulated MES and benchmark requests, is publicly accessible. • Chat with MES (CWM) replaces MES GUI, enabling human-computer interaction via linguistic commands. • Multi-step Dynamic Operations Planning and Execution breaks tasks into controllable, atomic operations for reliability. • New benchmark tests CWM with 55 crafted requests in a simulated garment manufacturing system. • CWM achieves 80% success rate, outperforming GPT-4’s 60%.