Software-defined self-learning control system for industrial robots by using reinforcement learning

Modern manufacturing requires systems that respond quickly and flexibly to changing production demands. Although software-based control systems that update programs to perform new tasks on existing equipment can partially address these requirements, they have limitations when applied to manufacturing equipment. Moreover, they face challenges in providing immediate responses to unexpected conditions and lack skilled human operators. This paper proposes a self-learning control system that integrates anomaly detection and reinforcement learning, enabling existing equipment to adapt to new tasks and equipment states through software updates. The proposed system uses a virtual environment that imitates various anomalous statuses in a self-learning approach. In this virtual environment, reinforcement learning models are trained for application to real equipment control and are stored with the status data where they occur. The anomaly detection algorithm launches different control models that were previously trained for the specific statuses being observed. A simple SCARA under different conditions was used to validate the proposed system. Anomaly statuses, such as the overcurrent status of the motor controller, are applied to the proposed system. The control system detects the status changes and switches the control model in 1.5 s without additional sensors such as an observation camera or joint encoders.