acceptodds
Under review as a conference paper at ICLR 2027

A PAC-Bayesian view of Generalisation for Physics-Informed Machine Learning

Abstract

Physics-informed machine learning (PIML) integrates mechanistic knowledge, typically in the form of partial differential equations (PDEs), into data-driven models. Despite strong empirical performance, its statistical generalisation properties remain poorly understood, particularly in the regression setting with unbounded losses. Existing analyses rely largely on approximation or stability arguments and do not fully capture how physical structure influences generalisation from finite data. In this work, we develop a PAC-Bayesian framework for PIML that provides high-probability generalisation guarantees in the presence of potentially unbounded losses. Our analysis leverages the structure of physics-informed objectives to derive component-wise bounds whose complexity scales with the input-gradient energy of each loss, revealing a direct link between physical regularity and generalisation. We instantiate this framework under population-level regularity conditions and introduce a PAC-Bayesian calibration procedure that yields computable input-gradient complexities while explicitly controlling rare large-gradient events through a residual-tail correction. We then adopt a multi-task perspective that jointly treats data fidelity, PDE residuals, initial conditions, and boundary conditions, leading to a refined certificate with a single PAC-Bayesian complexity penalty, thereby avoiding the looseness induced by controlling each component independently. Based on the resulting certificate, we further develop a bound-aware learning procedure that jointly promotes empirical accuracy, proximity to a physics-informed prior, and low input-gradient complexity. Experiments across six PDE benchmarks demonstrate substantially tighter certificates than conservative bounded-loss and sub-Gaussian alternatives, while targeted ablations characterise the effects of posterior-training data, calibration data, gradient envelopes, and hypothesis localisation on the resulting guarantees.

Primary area:
learning theory
Keywords:
PAC-Bayes bound, Physics-Informed Machine Learning, Generalization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_ZvHyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper considers the physics-informed learning setting through the lens of PAC-Bayesian generalization. The paper leverages the input gradients of the data fit loss and the physics-informed regularization to characterize the complexity. The main contribution of the paper seems that the certifications can be computed in a tight manner. It seems that there is no proof in the paper to characterize how physics improves prediction accuracy.

Critical strengths

The paper is really interesting, well-written and easy to follow. This idea of unsterstanding how the physical knowledge affects the complexity of the learning problem is very important and have been addressed by some papers in the literature that provide complexity dependent bounds for physics informed learning.

  1. “Physics-informed learning under mixing: How physical knowledge speeds up learning”
  2. “Physics-informed machine learning as a kernel method”.
  3. “Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Method”

I enjoyed reading this paper!

Critical weaknesses

The main drawback for me is that the paper explicitly leaves open how physical constraints affect generalization of the data fit loss. From the complexity dependent bounds in the papers mentioned above, it seems that it might be a way to extend excess risk bounds to generalization bounds, understanding how the physical knowledge reduces the complexity of the hypothesis class. I feel that the same reasoning could also be applied through the lends of PAC-Bayes, by optimizing over the priors. Maybe the authors could provide more intuitions on that.

In terms of scalability I would like to understand how this PAC-Bayes could be used for more complexity dynamics and physical lows. For example complex, coupled, and multi-output PDEs.

Questions for authors

See the weakness section

Reviewer_g4YTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper develops a PAC-Bayesian framework for physics-informed neural networks (PINNs), deriving computable generalisation bounds under population-level regularity assumptions. The framework links loss concentration to input-gradient complexity through a calibration procedure and an explicit gradient-tail correction, and introduces a joint analysis of multiple loss components to reduce repeated complexity penalties. The resulting bounds also motivate a bound-aware training objective. Experiments on six PDE benchmarks provide empirical support for the assumptions and demonstrate tighter certificates than the considered conservative bounded-loss and sub-Gaussian baselines.

Critical strengths
  1. The paper is clearly written and well organised, making the theoretical framework easy to follow.

  2. Building on existing PAC-Bayes tools, it provides useful extensions towards computable PIML certificates through calibration, gradient-tail control, and joint analysis of multiple loss components.

  3. The empirical study includes extensive diagnostics and ablations, with interesting observations such as broader gradient-energy distributions for PDE losses and the role of tail corrections in certificate looseness.

Critical weaknesses
  1. The ablations do not fully isolate the gains from empirical-gradient estimation and tail-aware control. A useful comparison would evaluate, on the same fixed posterior, a direct bounded-gradient concentration bound and the proposed clipped-gradient-plus-tail construction, using matched regularity constants and confidence levels. In particular, Theorem 16 of Casado et al. (NeurIPS 2024) already provides an empirical input-gradient PAC-Bayes bound under bounded-gradient assumptions and would be a relevant reference for this comparison. I have not fully assessed the feasibility of this comparison and would welcome clarification if differences in assumptions make it inappropriate.

  2. The magnitude of the benefit from posterior-specific calibration remains unclear. Could the authors quantify how much this step tightens the final certificate relative to a valid prior-based calibration strategy?

  3. While the computable certificates are valuable, their practical implications for PINN training are less clear. As an open question, what training decisions can they guide, and what benefits arise beyond physics-informed pretraining and standard fine-tuning?

Questions for authors

see weaknesses

Reviewer_8CBMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper builds generalization bounds for data error, PDE residuals, and initial/boundary conditions. It links the complexity of these bounds to the input gradient of the loss and uses calibration plus tail corrections to handle losses that might be unbounded. Experiments on six types of PDEs show that its certificates are usually tighter than the baselines. The main catch is that the guarantees rely on several population-level assumptions that you can't fully verify with finite samples, and the certificates can get loose if the gradient upper bound is too conservative.

Critical strengths

1.Handles potentially unbounded losses like squared error and PDE residuals.

2.Links generalization complexity to input gradient energy, which has a nice physical interpretation.

3.Treats rare large-gradient events separately.

4.Analyzes multiple losses jointly, avoiding redundant complexity penalties.

5.Tested on six types of PDEs, with analysis of factors like sample size.

Critical weaknesses

1.The core guarantee hinges on three population-level assumptions that can't be verified from finite samples alone.

2.If the gradient upper bound is loose, the certificate stays loose too.

3.Requires separate prior, posterior, and calibration datasets.

4.Not yet validated on complex coupled or multi-output systems.

5.A tighter certificate doesn't necessarily translate to better real-world decisions.

Questions for authors

1.How do you verify Assumption 1 holds for all admissible posteriors?

2.How do you determine a gradient upper bound that covers the entire input domain?

3.Is the calibration threshold robust to distribution shifts?

4.Is the extra computation for optimizing the certificate really worth it?

5.How do physical constraints actually improve the generalization of the data-fitting term itself?

Reviewer_RDjhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript establishes a PAC-Bayesian framework regarding generalization performance of PIML based on input-gradient regularity. Theoretically, the authors provide component-wise generalization bounds for data-fidelity, PDE, initial condition, and boundary condition risks, and further derive a joint multi-task bound to avoid repeated PAC-Bayesian complexity penalties across loss components. They also propose the Bound-Aware Learning algorithm, which uses the PAC-Bayes penalty in posterior optimization. In the experiments, the authors compare 4 certificates on 6 PDE benchmarks, considering both balanced and data-scarce settings, showing the tightness of their proposed method.

Critical strengths
  • This manuscript is clearly presented and well organized. The motivation, preliminary, theoretical framework, and experiment results are easy to follow.

  • The theoretical analysis is detailed and well developed, covering both component-wise and joint generalization bounds.

  • The authors conduct extensive experiments, which demonstrate that their proposed bounds outperform the bounded-loss and sub-Gaussian baselines regarding certificate tightness, while maintaining comparable test risks. And they furthur conduct ablation experiments to investigate the effects of calibration sample size, gradient envelops, localization radius, and posterior sample size on the bound tightness.

Critical weaknesses
  • Although the comparison with bounded-loss and sub-Gaussian certificates is useful, I'm concerned about the performance of the Bound-Aware Learning algorithm compared with standard PINN training, which only optimizes R^PIML,λ(θ)\hat{R}_{PIML,\lambda}(\theta) without PAC-Bayes regularization. Particularly the test risks, which help us better understand the benefit of this algorithm.

  • The empirical input-gradient PAC-Bayes bound was analyzed in Casado et al., 2024. Could the authors clarify the technical novelty or contribution of the proposed bound in the manuscript compared to the existing result, or more detailed discussion about the differences would help highlight the contribution of this work.

  • In lines 108-110, I think it would be clearer to explicitly clarify that $i \in \{ d,p,ic,bc \} $. This could make the notation easier to follow.

  • I think it's fair to consider unweighted risk in theoretical analysis, but I'm curious whether the joint bound can be extended to weighted PIML objectives, which better reflect practical PINN training. I'm also worried about whether the loss weights are adaptive during training.

Questions for authors

See Weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.