- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper tries to train a PINN from a Bayesian inference perspective. It proposes to use a factor graph to represent this problem, and runs a message-passing iteration to conduct sampling. The proposed method outperforms conventional Bayesian PINN methods in efficiency compared to HMC, and outperforms in accuracy compared to variational inference.
Critical strengths
- The paper proposes a well-motivated application of message passing to approximate posterior inference in B-PINNs. It is natural to represent the observation model and PDE constraints as two coupled subgraphs, and the moment matching + local linearization enable tractable (Gaussian-approximated) message updates. The experiments also suggest potential benefits of the proposed method.
Critical weaknesses
- The empirical gain can be solidified by covering more baselines that use accelerated MC/VI techniques in Bayesian inference. For example, existing works have used maximum a posteriori (MAP) + local second-order Gaussian approximation [1] or tailored MCMC sampling [2] to improve the trade-off between sampling efficiency and accuracy in Bayesian PINNs. Additionally, a more direct SGD baseline would be to use SGD for MAP estimation in the same Bayesian PINN rather than using SGD to train a deterministic PINN.
[1] MacKinlay et al., Gaussian Ensemble Belief Propagation for Efficient Inference in High-Dimensional, Black-box Systems, ICLR 2025. [2] Lin et al., Multi-variance replica exchange SGMCMC for inverse and forward Bayesian physics-informed neural network, Journal of Computational Physics 2022.
Questions for authors
- Can you clarify the main sources of the computational benefits of message passing over well-optimized MC or VI methods? How does the proposed method compare to more advanced B-PINN methods?
Curious question:
- When the problem setting is amortized inference (e.g., the model needs to condition on PDE boundary condition / paratemers) across a family of PDEs, conditional PINNs can directly have additional inputs. For Bayesian PINNs, this amortization does not seem straightforward. Particularly, can your methods be made amortized? Seems that learning a GNN on the factor graph in this case is a natural solution.
Thank you for taking the time to review our work! We appreciate your assessments of our submissions strengths as well as the questions you raised.
Any tables mentioned below are in our general comment.
Baselines: MAP + local Gaussian [1], SGD-MAP, tailored MCMC [2]. We added the MAP of the same Bayesian PINN (Adam + Gauss–Newton, your SGD-MAP) and Laplace at that MAP on all 130 runs (Table 1). Laplace matches HMC, so we claim no edge over it when the MAP is good. At ρ = 10 the MAP search ends on a worse mode in 13 of 20 runs, and 30 random restarts per run do not fix this. Message passing from the prior reaches HMC's mode in all 20 (Table 2; 19 with the code fixes in our reply to F6Fs). On the Gaussian initial condition at ρ = 10 the reverse holds (Table 5). We did not run GEnBP [1], which estimates Gaussian messages from ensembles of a black-box model, or replica exchange [2]; we discuss both. We see our approach's value where a good MAP is hard to find, as at ρ = 10.
Q1. Sources of the computational benefit. We need no MAP search, which HMC and Laplace start from, and we stop after at most 300 closed-form sweeps instead of up to 30,000 Adam steps plus Gauss–Newton. Table 2 gives the measured ratios: HMC at the smallest budget that passes its convergence gate takes 2.6–3.1× our time, Laplace 1.0–1.4×.
Amortization. Not covered here. With a fixed basis and collocation set, the factor graph is the same for every PDE parameter or boundary condition; only the factor potentials change. A GNN that takes those parameters as inputs could then learn the updates. We add it as future work. We think the fixed factor graph makes this a natural next step.