An Atlas of Physics-Informed Neural Network Failures: The Zugzwang Thesis

Standard continuous multi-layer perceptron (MLP) Physics-Informed Neural Networks (PINNs) are a prominent deep learning approach for scientific computing, yet they frequently fail on stiff partial differential equations (PDEs) and the underlying failure mechanisms remain incompletely quantified. We map this failure landscape through extensive controlled experiments across three stiff PDEs, isolating distinct mechanisms via spectral profiling, variance decomposition, and optimization diagnostics. We observe three primary phenomena: (i) Extreme loss landscape pathologies, where failed models converge to exceptionally sharp minima (λ_max ≈ 9.4 × 10⁷) compared to the flatter minima of successful architectures (λ_max ≈ 3.4 × 10³; noted as a directional finding due to confidence interval (CI) overlap at n=5). (ii) Structural factors heavily dominate hyperparameter tuning in determining outcome variance (η²=0.673 vs 0.0023 for Burgers, 1.7:1 ratio for advection). (iii) Spectral failures at high stiffness exhibit poor initialization conditioning (Neural Tangent Kernel condition number κ_NTK ~ 10⁹ to 10¹¹) and extreme gradient conflicts (individual measurements as negative as −0.9996 (≈ −1.00, indicating near-exact gradient opposition)), acting independently from other failure modes (lagged cross-correlation r=0.630). Crucially, standard scalar metrics obscure these physical violations. Two distinct failure configurations exhibit highly divergent optimization trajectories and conservation violation profiles. For instance, Gradient Pathology and Collocation Starvation both result in catastrophic failure, yet they shatter the physics in distinct ways, exhibiting significantly different relative error magnitudes (0.888 vs 0.660) and mass conservation leakages (C₁: 27.24 vs 7.75). These results demonstrate that PINN failures stem from fundamental architectural bottlenecks rather than suboptimal tuning. We term this the Zugzwang Thesis: standard PINNs encounter multiple, independent failure mechanisms simultaneously under stiffness, yet no single tested intervention provided a structurally complete remedy.