Intelligent low carbon reinforced concrete beam design optimization via deep reinforcement learning

Reinforced concrete (RC) beam design currently faces significant challenges from the substantial carbon footprint of cementitious materials and the lack of practical automated tools for simultaneous structural-environmental optimization. To address this, this paper proposes an innovative two-stage framework by utilizing artificial neural networks (ANNs) and deep reinforcement learning (RL) to automate the design of sustainable and low-carbon RC beams. In the first stage, following a comprehensive analysis of 14 machine learning algorithms, an ANN was selected for its superior predictive accuracy. The trained ANN effectively predicts concrete compressive strength (coefficient of determination, R² ≈ 0.85) and carbon dioxide (CO₂) emissions (R² ≈ 0.99), critical parameters for subsequent optimization, exhibiting a loss function value of 0.15 and a mean absolute error of 0.38. The second stage involves the decision-making process for designing RC beams through deep RL, utilizing a Proximal Policy Optimization (PPO) as an agent. The agent operates within a 13-dimensional parametric action space, encompassing geometric and material composition variables, and interacts with a 26-variable state space to balance structural integrity with environmental sustainability. A customized RL environment was created to optimize designs for minimal CO₂ emissions and evaluate compliance with ACI 318 - 19 flexural design criteria. The resulting framework demonstrates comprehensive sustainability achievements, with comparative benchmarking showing PPO-optimized designs yield 43.35-75.04% lower CO₂ emissions than those from an Advantage Actor-Critic (A2C) agent, alongside automated ACI 318 - 19 code compliance, optimized utilization of supplementary cementitious materials (SCMs), and improved structural efficiency through intelligent geometric parameter selection. The code is available as open source, and a web-based interface facilitates the dissemination of research outcomes.