Generating synthetic data for neural operators

Recent advances in the literature show promising potential of deep learning methods, particularly neural operators, in obtaining numerical solutions to partial differential equations (PDEs) beyond the reach of current numerical solvers. However, existing data-driven approaches often rely on training data produced by numerical PDE solvers (e.g., finite difference or finite element methods). We introduce a “backward” data generation method that avoids solving the PDE numerically: by randomly sampling candidate solutions <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>u</mml:mi> <mml:mi>j</mml:mi> </mml:msub> </mml:math> from the appropriate solution space (e.g., <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:msubsup> <mml:mi>H</mml:mi> <mml:mn>0</mml:mn> <mml:mn>1</mml:mn> </mml:msubsup> <mml:mrow> <mml:mo>(</mml:mo> <mml:mi>Ω</mml:mi> <mml:mo>)</mml:mo> </mml:mrow> </mml:mrow> </mml:math> ), we compute the corresponding right-hand side <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>f</mml:mi> <mml:mi>j</mml:mi> </mml:msub> </mml:math> directly from the equation by differentiation. This produces training pairs <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mo>(</mml:mo> <mml:msub> <mml:mi>f</mml:mi> <mml:mi>j</mml:mi> </mml:msub> <mml:mo>,</mml:mo> <mml:msub> <mml:mi>u</mml:mi> <mml:mi>j</mml:mi> </mml:msub> <mml:mo>)</mml:mo> </mml:mrow> </mml:math> by computing derivatives rather than solving a PDE numerically for each data point, enabling fast, large-scale data generation consisting of exact solutions. Experiments indicate that models trained on this synthetic data generalize well when tested on data produced by standard solvers. While the idea is simple, we hope this method will expand the potential of neural PDE solvers that do not rely on classical numerical solvers for data generation.