DQN-Based Pre-Optimization for Dual-Scale Collaborative Topology Optimization of Anisotropic Materials

Traditional topology optimization methods often face challenges such as slow convergence, high sensitivity to initial structures, and limited exploration of the design space when dealing with multi-physics coupling problems. To address these challenges, this study proposes an efficient design framework integrating reinforcement learning and topology optimization. The framework first employs a Deep Q-Network (DQN) agent to dynamically adjust penalty factors, accelerating the convergence process, and uses its pre-optimization results as the initial conditions for the Bidirectional Evolutionary Structural Optimization (BESO) method, thereby enhancing optimization efficiency and structural performance. By introducing an anisotropic material model, the design space is expanded, further unlocking the potential for structural lightweighting. On this basis, a dual-objective optimization strategy for mechanical compliance and thermal compliance is adopted, enabling the final structure to adapt to various physical working conditions. Finally, the optimal design is extended from two-dimensional to three-dimensional, facilitating subsequent manufacturing and verification. Numerical examples demonstrate that compared with traditional methods, the proposed pre-optimization method achieves a 22.463% reduction in structural compliance and improves thermal management performance. The framework demonstrates robust convergence across different boundary conditions (MBB and cantilever beams) and expands the design space through anisotropic microstructures, offering a practical solution for multi-physics lightweight design.