Deep reinforcement learning based topology optimization of triangular meshes

• Intelligent feature-preserving topology optimization of triangular meshes. • Multi-head GAT for dynamic topology feature encoding. • A set number of fixed-radius regions balances optimization speed and accuracy. • Reward function integrating topology quality and fidelity. Triangular meshes play a crucial role in finite element analysis, especially when modeling complex geometries with irregular boundaries and fine geometric details. However, mesh quality directly impacts analysis accuracy and efficiency. Existing mesh generation and optimization methods are still unable to meet high-quality requirements, and most mesh topology optimization approaches rely on heuristic strategies and manual expertise, making them vulnerable to local optima. To address this issue, this paper proposes a deep reinforcement learning-based topology optimization framework for triangular meshes. This framework constructs a multi-head graph attention network (GAT) that integrates vertex topological properties with edge geometric features. It designs a reward mechanism that combines irregularity improvement and geometric feature preservation, guiding the agent to independently explore optimal topological adjustment strategies through self-play without prior knowledge. This approach preserves geometric features while approximating the optimal vertex degree distribution and effectively reducing mesh irregularity. Experimental results demonstrate that the method is effective and robust in local adaptive topology optimization, providing new insights into high-quality mesh topology optimization.