Blending Data and Knowledge for Process Industrial Modeling Under Riemannian Preconditioned Bayesian Framework

Integrating graph neural networks (GNNs) with variational inference (VI) provides a promising direction for blending structured prior knowledge with observational empirical data for data-driven industrial process modeling. However, this task requires inference of the normalized adjacency matrix (NAM), where each row is normalized to be non-negative and to sum to one, matching the support of Dirichlet distribution. This requirement presents two main technical challenges: 1) intractable Kullback-Leibler (KL) divergence optimization between Dirichlet distributions, and 2) constrained optimization for standard- gradient-descent-based neural network parameter optimization. To handle issue 1), we first formulate the inference of the NAM as a differential equation simulation problem and derive an easy-to-implement expression to iteratively improve the KL divergence without explicitly computing it. Based on this, to alleviate issue 2), we involve Riemannian optimization to precondition this simulation procedure, which ensures that the inferred NAM conforms to the row-normalization constraint. After that, we collectively designate these approaches for NAM inference as Preconditioned-Simulation-Induced Variational Inference (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\psi$</tex-math></inline-formula>-VI), and provide theoretical guarantees of convergence. On this foundation, we propose a new graph neural network architecture, the Preconditioned-Simulation-Induced-based Variational Graph Neural Network (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\psi$</tex-math></inline-formula>-VGNN) for industrial process modeling. Finally, we validate the efficacy of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\psi$</tex-math></inline-formula>-VGNN through comprehensive experiments on industrial modeling tasks.