RES-PPO: A Residual-Augmented Proximal Policy Optimization Approach for Integrated Optimization of Chemical Production, Transportation, and Sales

Chemical enterprises encounter challenges stemming from inadequate resource integration, operational inefficiencies, and suboptimal production-transportation-sales coordination. To mitigate these limitations, this study introduces a novel Residual-augmented Proximal Policy Optimization (Res-PPO) framework for integrated optimization of chemical production-transportation-sales. The proposed model holistically incorporates operational factors—including market demand fluctuations, production capacity constraints, transportation limitations, and dynamic inventory requirements—to maximize profitability through synchronized cross-functional decision-making. Key methodological innovations include: 1) Integration of residual blocks into the PPO network architecture to enhance gradient propagation; 2) Advanced optimization techniques combining Cosine Annealing with Adam (CA-Adam) for adaptive learning rates; 3) A Staged Entropy Decay (SED) mechanism for balanced exploration-exploitation tradeoffs. Experimental results demonstrate that RES-PPO achieves superior performance compared to conventional approaches (CPLEX, GUROBI), baseline algorithms (DQN), and original PPO, showing <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$6 \%-76 \%$</tex> higher profit. In addition, compared to DQN and original PPO, the RES-PPO algorithm achieving a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$53 \%-64 \%$</tex> decrease in runtime, a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$68 \%-69 \%$</tex> reduction in convergence iteration steps, a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$10 \%- 16\%$</tex> lower peak memory usage, and exhibits greater training stability.