Towards optimised perishable inventory systems: integrating time-temperature indicators with deep reinforcement learning

Time-Temperature Indicators (TTIs) can improve the control of perishable temperature-regulated products in cold-chain operations. Companies monitor product quality and temperature over the full cold-chain process, and TTIs provide visual signals indicating product freshness. This study develops a periodic-review temperature-sensitive perishable inventory model that integrates TTIs and Deep Reinforcement Learning (DRL) within a high-dimensional Markov decision process. We implement Proximal Policy Optimization (PPO) to learn ordering policies that capture multi-period inventory aging and demand uncertainty. Through numerical experiments, we compare PPO against a base-stock policy and a projected inventory level policy in TTI-enabled and non-TTI settings. PPO achieves the highest mean performance overall and in almost all tested configurations, with the PPO advantage increasing as deterioration becomes faster. Comparing systems with and without TTIs shows profit gains up to about 45%. The value of TTI information depends on deterioration rate, green/yellow demand mix, and whether replenishment responds to the green/yellow stock mix. TTIs are more beneficial when the deterioration-generated stock mix is close to demand. Profit may decrease when this alignment breaks down, either because too many items move to yellow while demand remains mainly green, or because yellow demand arises before enough stock has moved from green to yellow.