Deep Reinforcement Learning-Based Optimization for the Shape of an Adhesive Pillar With Enhanced Adhesion Strength

Abstract Biological adhesive systems with nano- and micro-structured features, like those found in geckos, beetles, and spiders, have been extensively studied for their reversible adhesion properties. Various optimization studies, including parametric and machine learning-based approaches, have sought the optimal pillar shape but often face limitations due to the trade-off between exploring the vast design space and getting stuck in local minima. To balance data efficiency, computational time, and optimization accuracy, we introduce a deep reinforcement learning (DRL)-based incremental shape optimization scheme. This method discovers the optimal geometry of adhesive pillars with sharp and truncated edges, enhancing adhesion strength based on interfacial stress distribution. Free-form pillars are generated using a Bézier curve, with control points representing states and their adjustments considered actions in deep reinforcement learning. The agent interacts with a finite element method-based environment to learn policies for optimal designs, achieving robust performance across different initial geometries through efficient reward shaping. The similarity between optimal shapes allows efficient transfer of the optimal policy from sharp-edged to truncated-edged pillars. Both optimized pillars feature a concave tip and convex stalk to mitigate stress concentration. Our method demonstrates competitive performance compared to deep learning surrogate model-based and Bayesian optimization methods in terms of optimization accuracy and computational time, offering a promising solution for complex shape optimization problems.