Dynamic Control of Cardboard-Blank Picking by Using Reinforcement Learning

Abstract Standard packaging lines with high output rates often struggle when dealing with uncertainties in the conditions of the handled materials. This paper focuses on a piece of machinery in an automatic packaging line, namely, an automated apparatus that extracts cardboard blanks from a buffer and transfers them to the next section through suction cups. In this context, the success of the operation depends on various controllable parameters, disturbances, and time-dependent variables, whose mutual relationships are not easily identifiable and whose understanding has so far been entrusted to the experiential knowledge of human operators. Currently, drops in picking success rates require the machine to be stopped and operators to intervene on-site, making use of their expertise to identify the issue and recalibrate the machine. To address the problem, this paper presents an artificial-intelligence-enabled controller, capable of continuously and autonomously recalibrating the apparatus and compensating for disturbances, in order to avoid missed or incorrectly picked cardboard blanks. In particular, this work exploits experimental data to build a model of the system, on which a reinforcement-learning algorithm is trained. The controller is tasked with regulating the controllable parameters while monitoring process variables. The developed agent is tested on the real apparatus to assess its performance.