Multi-Objective Constrained Deep Q-Networks for Supplier Selection and Order Allocation in Sustainable Supply Chains

Abstract Sustainable sourcing requires sequential decisions that couple discrete supplier admission, integer order quantities, stochastic lead times, inventory, and competing economic, environmental, and service objectives. We formulate this setting as a preference-conditioned constrained Markov decision process with state-dependent, mixed-integer action feasibility and distributional tail-risk requirements. We develop ParetoShield-Q, a multi-objective deep Q-learning formulation that uses an augmented Chebyshev action functional over distributional cost-to-go estimates and a completion-feasibility oracle to construct executable supplier allocations. The same constrained Bellman operator jointly determines which objectives are optimized, which downstream risks are penalized, and which supplier--quantity assignments are legally selectable. We establish conditional one-step feasibility and finite-candidate Pareto-efficiency properties, and provide an evaluation protocol based on the public M5 demand data and a public supplier-attribute dataset. The study presents seven quantitative tables and eight vector figures spanning multi-objective performance, preference control, cost composition, operational feasibility, ablations, disruption response, and computational scaling. The visualizations reproduce the tabulated numerical summaries without constructing unobserved run-level training traces. Independent reproduction requires the source training runs, scenario configurations, and optimizer logs.