QR-DeepONet: resolve abnormal convergence issue in deep operator network

Abstract Deep operator network (DeepONet) has been proven to be highly successful in operator learning tasks. Theoretical analysis indicates that the generation error of DeepONet should decrease as the basis dimension increases, thus providing a systematic way to reduce its generalization errors (GEs) by varying the network hyperparameters. However, in practice, we found that, depending on the problem being solved and the activation function used, the GEs fluctuate unpredictably, contrary to theoretical expectations. Upon analyzing the output matrix of the trunk net, we determined that this behavior stems from the learned basis functions being highly linearly dependent, which limits the expressivity of the vanilla DeepONet. To address these limitations, we propose QR decomposition enhanced DeepONet (QR-DeepONet), an enhanced version of DeepONet using QR decomposition. These modifications ensured that the learned basis functions were linearly independent and orthogonal to each other. The numerical results demonstrate that the GEs of QR-DeepONet follow theoretical predictions that decrease monotonically as the basis dimension increases and outperform vanilla DeepONet. Consequently, the proposed method successfully fills the gap between the theory and practice.