Learning Optimal Reprocessing Policies for Degradable Inventory Systems: A Multi‐Armed Bandit Approach

ABSTRACT In this paper, we study the problem of online learning for a threshold‐based inventory control policy in degradable inventory systems with reprocessing capabilities, where the demand is unknown but independent and identically distributed (i.i.d.) across periods. A key feature of our model is that degraded items can be held as inventory rather than being immediately salvaged or reprocessed, providing additional flexibility in operational decisions. Unlike traditional models that assume prior knowledge of the demand distribution, we develop an adaptive learning algorithm to determine the optimal reprocessing and production decisions without such information. Our analysis is motivated by medical supply chains where products like oxygen cylinders degrade over time but can be reprocessed through sterilization and refilling. The model incorporates key operational features including: (1) discrete quality degradation of inventory over periods, (2) the option to hold degraded items in inventory before reprocessing, and (3) the trade‐off between production, reprocessing, and inventory holding costs. Through a novel notion of generalized multi‐modularity tailored to our state‐action structure, we establish the optimality of a state‐dependent threshold policy with state‐independent threshold parameters, governing both reprocessing and production decisions. When demand is unknown a priori, we propose an online learning algorithm and prove that the algorithm achieves a cumulative regret of . This work contributes to both production‐inventory coordination and online learning literature by providing: (1) structural analysis using a generalized multi‐modularity framework to characterize the optimal policy in the setting with known demand, (2) the first learning‐theoretic framework for reprocessable and degradable inventory systems where degraded items can be strategically held before reprocessing, and (3) theoretical performance guarantees through regret analysis. The methodology applies to various industrial settings where products degrade discretely over time and can be held in degraded states before being reprocessed or salvaged.