Domain Decomposition of Large Neural Network Surrogate Models

Data-driven neural networks (NNs) have gained significant attention across engineering disciplines, particularly in design optimization and experimental settings, where they are widely used to construct surrogate models for high-dimensional regression problems. Despite their power as global approximators, neural networks often struggle to accurately capture local features without relying on a large number of trainable parameters and training data points, resulting in increased training time. To address these limitations, in this paper we propose domain decomposition methods (DDMs), which divide the input feature space into multiple local subdomains, each modeled by a simpler NN, trained in parallel. Interface constraints are introduced in the local loss functions to enforce continuity between subdomains. They are enforced with two different approaches: by utilizing Lagrange multipliers or augmented Lagrange multiplier methods. Compared to unconstrained approximations, both methods significantly improve continuity across subdomain interfaces. For a 2D and a 3D problem, computational time and accuracy are investigated across varying numbers of subdomains to identify optimal partitioning strategies. The use of DDMs improves approximation accuracy in local regions with smaller number of parameters when compared to standard global NN training. In terms of convergence, the augmented Lagrange method outperforms the standard Lagrange formulation by converging faster due to lower convergence requirements, albeit with a slightly lower accuracy. Overall, these results highlight the augmented Lagrange method as a promising DDM approach for training efficient and scalable NN surrogate models.