- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an Extended Fiducial Inference (EFI) framework for structurally constrained nonlinear regression with unknown and non-Gaussian observation noise. The key idea is to use a hypernetwork to map individual observation and noise variables to the parameters of another (regression) network, which enables the construction of confidence and prediction intervals. Under a low-dimensional representation assumption, the authors establish the theoretical guarantee and experiments on synthetic regression problems shows competitive accuracy and near-nominal coverage.
Critical strengths
The motivation for uncertainty quantification in structurally constrained nonlinear regression is important and well justified.
Critical weaknesses
W1. My biggest concern is the methodological novelty. It seems to me the proposed method combines existing EFI for PINNs with nonparametric-error EFI method, so the methodological contribution looks somewhat incremental.
W2. Another central concern is related to Assumption 4.1, which requires the constrained function class to admit a low-dimensional representation, which seems to be a relatively strong assumption. How this can be justified for general structural constraints, e.g periodicity and monotonicity. If the assumption does not hold, to what extent can the theoretical results still justify the proposed uncertainty quantification?
Questions for authors
Q1. Please follow the ICLR format requirement for abstract, single paragraph. The keywords part should be deleted.
Q2. Discussion with probabilistic function regression methods is kind-of limited. e.g. Gaussian Process Regression (GPR), Deep GPs, and Neural Processes. In particular, GPR can also incorporate differential constraints and provide uncertainty estimates . The authors could at least include a controlled comparison with GPR .
Response to W1
Our contribution is to identify and formalize how structural restrictions can connect dense nonlinear models to flexible-noise inference through a low-dimensional representation. Please see our Shared Response on Methodological Novelty for the distinction from prior EFI work and the supporting evidence.
Response to W2
A sufficient condition for Assumption 4.1 is a linear differential structural restriction Lf=r with dimker(L)=q<n. For example, the continuous 1-D Poisson equation with fixed forcing gives u(x)=u0(x)+ϕ1+ϕ2x, with q=2, where u0 is a particular solution.
Weaker restrictions such as periodicity and monotonicity may leave a high-dimensional admissible class, for which we have not established Assumption 4.1. Here, the narrow neck provides a learned low-rank approximation in parameter space. When dh<q, the proposed method quantifies the uncertainty of θ in a low-dimensional space. Note that uncertainty of θ can still be correctly quantified within this restricted function class, provided that the w-network is sufficiently expressive. The periodic experiment demonstrates near-nominal coverage, while the monotonic ablation (Table E.6) shows only a small change in CI coverage as dh increases from 4 to 16, with similar interval widths.
Assumption 4.1 restricts the flexibility of the regression function so that inference with an unspecified noise distribution can be justified under the applicable reduced-model EFI conditions. Without sufficient restrictions, the regression function can absorb observational noise, and the theoretical UQ guarantees no longer follow.
Response to Q1
Thank you for pointing this out. We have revised the abstract to a single paragraph and removed the keywords section.
Response to Q2
We added two GP comparisons incorporating the same structural information: a periodic GP with a Matérn-3/2 kernel on a sine–cosine embedding of the known period, and a Poisson operator GP with a Matérn-5/2 kernel following [1] (Table 2).
Under Gaussian noise, both GPs and EFI perform well, with narrower CIs for the periodic GP. GP can provide efficient inference when the chosen kernel captures the regression function well and the noise assumption is appropriate. Under bimodal noise, the GPs retain near-nominal CI coverage, but their Gaussian-likelihood PIs over-cover. EFI achieves lower MSE and narrower CIs and PIs, with coverage close to nominal. These GP baselines encode structure through tailored kernels and use a Gaussian likelihood. EFI accommodates structural constraints and an unspecified noise distribution within the same inferential framework.
[1] Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 348:683–693, 2017.
Table 2. GP versus EFI: means over 100 replications.