Conditional Entropy-Based Sequential Decision-Making for AI Adoption in Manufacturing: A Reinforcement Learning Approach

Most small- and medium-sized manufacturers face challenges in adopting artificial intelligence (AI) in production systems due to limited domain expertise and challenges in making interrelated decisions. This decision-making process can be characterized as sequential decision-making (SDM), in which guidance on the decision order is valuable. This study proposes a data-driven SDM framework to identify an effective order of key decision elements for AI adoption, aiming to rapidly reduce uncertainty at each decision stage. The framework employs a Q-learning-based reinforcement learning approach, using conditional entropy as the reward function to quantify uncertainty. Based on a review of 55 studies applying AI to milling processes, the proposed model identifies the following decision order that minimizes cumulative uncertainty: sensor, data collection interval, data dimension, AI technique, data type, and data collection period. To validate the model, we conduct simulations of 4000 SDM episodes under rule-based constraints using the number of corrected episodes as a performance metric. Simulation results show that the proposed model generates decision orders with no corrections and that knowing the relative order between two elements is more effective than knowing exact positions. The proposed data-driven framework is broadly applicable and can be extended to AI adoption in other manufacturing domains.