acceptodds
Under review as a conference paper at ICLR 2027

Multi-Fidelity PDE Learning with Adaptive Spectral Neural Processes

Abstract

Efficiently learning solution operators for parametric partial differential equations (PDEs) is difficult when high-fidelity simulations are scarce and expensive, even though low-fidelity approximations are often abundant. We introduce Multi-Fidelity Physics-Informed Neural Processes (MF-PINP), an adaptive spectral framework for multi-fidelity PDE learning that models each fidelity as an auto-regressive correction to the preceding level while imposing governing equations at resolution-dependent tolerances. Learnable spectral position encodings equipped with sparsity-inducing log-uniform priors select frequencies suited to each resolution and capture multi-scale solution structure. Physics-informed cross-fidelity attention then fuses observations according to spatial proximity and PDE residuals, allowing the model to discount unreliable coarse information. We derive PAC-Bayes generalization bounds connecting spectral complexity to model capacity and calibration bounds for uncertainty propagated through the fidelity hierarchy. Experiments across benchmark parametric PDEs show that MF-PINP improves predictive accuracy, maintains well-calibrated uncertainty, and substantially reduces the need for expensive high-fidelity observations.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Multi-Fidelity Learning, Physics-Informed Neural Processes, Parametric Partial Differential Equations, Adaptive Spectral Representations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Ui2bOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose MF-PINP, a multifidelity neural operator that leverages abundant low-fidelity observations along with sparse high-fidelity ones to predict full-field solution data. Learnable spectral embeddings combined with “physics-informed” cross-attention adaptively downweight low-fidelity information in the presence of poor accuracy. PAC-Bayes theory is used to establish a probabilistic generalization bound on the spectral encoding, and a bound for controlling calibration error in the multifidelity prediction is also presented. However, it is hard to see a use case for this method since it requires observations of the field it is trying to predict, and it is unclear how the theoretical analysis is useful in practice. The paper is also hard to read, missing about 15 years of related work, and missing a stated benchmark. In general, I think its deficiencies outweigh its benefits and (weakly) recommend rejection.

Critical strengths

The idea of combining multifidelity methods with neural operators is interesting. The authors’ theoretical analysis is also appreciated.

Critical weaknesses

The use case of the authors’ method is not clear to me. When would observations of a high-fidelity solution on a regular grid be available without access to the entire solution? Typical physics-informed neural operators do not require solution data for the field they are trying to predict, while this work does. It is also hard to understand why a huge family of related work in multifidelity methods was omitted from the paper’s discussion (e.g., [1] and the survey [2] with over 200 references). These works tackle the same problem, and similarly do not require such detailed information. While I appreciate the analysis in section 4, it is unclear how it relates to the benchmarks the authors are actually implementing. Can you validate the bounds you have derived? It is claimed that table 4 validates theorem 4.2, but I don’t see how. For example, it’s not clear to me that the necessary assumptions are enforced. Speaking of experiments, the 2D Navier-Stokes benchmark mentioned in section 5 is not included in the body, and “vorticity formulation at Re=500” is so vague it is almost meaningless. Finally, the premise of the paper is misleading: traditional numerical solvers certainly do not “require solving [these] systems from scratch”, as any production code employs warm-starting from a converged solution (among other tricks) to accelerate convergence.

[1] Gorodetsky, Alex A., et al. "A generalized approximate control variate framework for multifidelity uncertainty quantification." Journal of Computational Physics 408 (2020): 109257.

[2] Peherstorfer, Benjamin, Karen Willcox, and Max Gunzburger. "Survey of multifidelity methods in uncertainty propagation, inference, and optimization." SIAM Review 60.3 (2018): 550-591.

Questions for authors

See weaknesses.

2 comments

Thank you for the review. MF-PINP reconstructs fine fields from partial observations by using PDE residuals to decide how much coarse information to transfer.

Q1. Practical use and test-time observations

Reply A coarse heat-transfer simulation can capture the broad temperature pattern, while accurate probes supply local observations. MF-PINP combines the two to predict fine values at unobserved locations. Our benchmarks emulate partial observations by partitioning simulated fields into context and targets. Table 6 also measures training data efficiency. With 500 coarse and five fine training simulations, the 2D Poisson error drops from 6.93e-4 to 2.47e-4 versus independent fidelity models.

We fix the trained model, test instances, coarse context, and targets. Fine context sets are nested and disjoint from the targets. The 0-point row uses the coarse prediction. Increasing fine context from 8 to 128 points lowers mean error by 92.5%.

Test-time observation budget. 2D Poisson u6u_6, relative L2L^2 error (mean ± SD, TBD seeds)

PredictionFine points per test instanceError
Coarse prediction05.20e-3 ± 5.41e-4
MF-PINP83.71e-3 ± 4.68e-4
MF-PINP321.12e-3 ± 1.65e-4
MF-PINP1282.77e-4 ± 3.73e-5

Q2. Relationship to classical multifidelity methods

Reply Gorodetsky et al. [1] use correlated model evaluations to reduce the variance of an expectation estimator, such as mean drag under uncertain inputs. MF-PINP estimates the field for an individual instance from partial observations, with uncertainty at query locations. We weight coarse observations by pointwise PDE residuals and use their mean to gate fusion across fidelities. In a five-seed Allen–Cahn u1u_1 ablation, supplying the residual to the gate lowers mean error by 56.5%. We will cite [1] and the survey [2] and explain how conditional field reconstruction complements multifidelity model management.

Q3. Role of the theory and evidence in Table 4

Reply For i.i.d. PDE tasks and a bounded task loss, the revised PAC-Bayes bound relates empirical risk to population risk for a randomized predictor. It retains the full parameter KL under a data-independent prior. Assessing its tightness requires the bounded loss and the KL of the weight posterior. The benchmarks evaluate predictive accuracy and data efficiency directly.

Table 4 reports 93.8% empirical coverage at nominal 95% and an ECE of 0.024. The exact variance identity for our correction architecture includes covariance between the inherited prediction and the learned correction. This term can remain after marginalizing the shared latent. Joint latent draws preserve this dependence. We will present the identity in place of the claimed calibration rate and report Table 4 as empirical evidence.

Q4. Navier–Stokes benchmark

Reply Appendix C.5, Table 11 reports 2D vorticity at Re=500\mathrm{Re}=500 across 32232^2, 1282128^2, and 2562256^2 grids. The benchmark tests fusion across three fidelities in a nonlinear flow. MF-PINP lowers relative L2L^2 error by 21.6% versus MFRNP.

MethodRelative L2L^2 errorCoverage (nominal 95%)
FNO4.12%—
MFRNP3.15%86.7%
MF-PINP2.47%92.1%

We will highlight these results in Section 5 alongside the equation, domain, forcing, initial and boundary conditions, and per-level discretization and context settings. Section 3 will summarize training inputs, test inputs, and prediction targets.

Q5. Warm starts in numerical solvers

Reply We will clarify in the introduction that nearby converged solutions can initialize new solves. In this sweep, warm starting reduces the fine solver's online time from 131 to 48.6 s per instance. MF-PINP takes 5.54 s online at a relative error of 2.77e-4. The warm-started solver reaches 2.15e-6 and has the lower amortized total time over 50 queries. The table reports the achieved error and cost for each method.

Warm-start comparison. 2D Poisson parameter sweep (50 queries)

MethodRelative L2L^2 errorOnline ms/instanceTotal ms/instance
Fine solver, cold start3.77e-61.31e51.42e5
Fine solver, warm start2.15e-64.86e44.93e4
MF-PINP (128 fine points)2.77e-45.54e34.07e5

References

[1] Gorodetsky et al. A Generalized Approximate Control Variate Framework for Multifidelity Uncertainty Quantification. [Journal of Computational Physics 408, 109257, 2020].

[2] Peherstorfer et al. Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization. [SIAM Review 60(3), 550–591, 2018].

I appreciate the authors' work in acknowledging my concerns. Since I still have concerns about the utility of the approach and substantial doubts about the relevance of the theory, I maintain my score of "weak reject".

Reviewer_V34gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MF-PINP, a multi-fidelity neural process for parametric PDEs. Fidelity levels are modeled as auto-regressive corrections of the preceding level. The approach combines different modeling choices: feature engineering through spectral position encodings with a log-uniform sparsity prior; and architectural design choices: physics-informed cross fidelity attention and hierarchical physics regularization. The authors also provide theorems containing a PAC generalization bound and a Berry--Esseen style calibration bound. The approach is evaluated on five PDE families, where results are compared to prior approaches and a component ablation is carried out.

Critical strengths
  1. Evaluation: The experimental setup appears to be fair and thorough. The authors include five PDE families and several baselines, both multi- and single fidelity. I also particularly like the honest parameter ablation study.
  2. Residual-gated cross fidelity attention: This appears to be a genuinely useful design idea and methodological contribution of this work. Unfortunately, the paper does not establish its magnitude properly.
Critical weaknesses

Unfortunately, the theory does not hold as written.

  1. Theorem 4.1: The KL bound in Eq. (31), KL(QNN∥PNN)=O(dzlog⁡n)\mathrm{KL}(Q_{\mathrm{NN}}\|P_{\mathrm{NN}}) = O(d_z \log n), is asserted from the Rademacher bound in Eq. (29), but Rademacher complexity is not a KL divergence, and the derivation drops the spectral-norm factor BB, the product ∏ℓdℓ\prod_\ell d_\ell, CϕC_\phi, and the Q…\sqrt{Q\ldots} term. The truncation rate εtrunc(Q)=O(Q−1/2)\varepsilon_{\mathrm{trunc}}(Q) = O(Q^{-1/2}) also requires s≥dxs \ge d_x (Eq. 35), while the stated assumption is only s>dx/2s > d_x/2.

  2. Theorem 4.2: The variance recursion doesn't work as stated. Conditional independence δℓ⊥u^ℓ−1∣z\delta_\ell \perp \hat{u}_{\ell-1} \mid z gives only E[Cov(δ,u^∣z)]=0\mathbb{E}[\mathrm{Cov}(\delta,\hat{u}\mid z)] = 0; the unconditional covariance still contains Cov(E[δ∣z],E[u^∣z])\mathrm{Cov}(\mathbb{E}[\delta\mid z], \mathbb{E}[\hat{u}\mid z]), which need not vanish because both conditional means depend on the same latent zz (take δ=z\delta = z, u^=z+ε\hat{u} = z + \varepsilon). Eqs. (43)/(45) drop exactly this term. The Berry--Esseen step is then applied to a nonlinear pushforward through the decoder and for ℓ>0\ell > 0 to a softmax-attention latent rather than a mean of Nmin⁡N_{\min} independent variables. Hence the O(1/Nmin⁡)O(1/\sqrt{N_{\min}}) calibration rate does not follow, and Nmin⁡=50N_{\min} = 50 cannot explain the claimed benefit of adding low-fidelity data.

  3. The contribution is thinner than the framing suggests. The individual ingredients are known: learnable Fourier features, log-uniform/ARD sparsity, Kennedy--O'Hagan-style auto-regressive correction, residual-based attention, and PAC-Bayes meta-learning. The closest work is mis-cited: "MFPC-Net Liu & Wang (2019)" is actually Wang & Lin (2020, arXiv:2010.01378), which already combines a neural process, multi-fidelity data, physics constraints, and predictive uncertainty. MFR-PINP (arXiv:2511.08231) is also uncited, and GP-HM's log-domain/Jeffreys-prior frequency sparsity is not credited. The claim that "none of the above methods achieve [...] simultaneously" is therefore unsupported. The residual-gated cross-fidelity fusion is the defensible novelty and should be foregrounded.

  4. Several headline numbers do not match the tables. The "over 127×127\times" LSE claim at line 456 is wrong: the ablation (Table 3, σ=100\sigma = 100) gives 8.93×10−6/3.85×10−7≈23.2×8.93\times 10^{-6} / 3.85\times 10^{-7} \approx 23.2\times. The data-generation saving is 9.9×9.9\times by the paper's own costs, not "5×5\times"; the PICA "45%45\%" is a +157%+157\% change in Table 5 (1.83×10−5→7.12×10−61.83\times 10^{-5} \to 7.12\times 10^{-6}); and the interpolation "within 15%15\%" is 2525--33%33\%. Also, the table seem to report single seed results. Given the very narrow improvements e.g. over GP-HM in Table 2, this looks rather suspicious.

Questions for authors
  1. You cite "MFPC-Net Liu & Wang (2019)", but the paper named MFPC-Net is Wang & Lin (2020, arXiv:2010.01378), which already combines a neural process, multi-fidelity data, physics constraints, and predictive uncertainty. Which implementation did you benchmark? How do you justify the claim that "none of the above methods achieve [these] simultaneously" given Wang & Lin (2020), the uncited MFR-PINP (arXiv:2511.08231), and GP-HM's existing log-domain/Jeffreys-prior frequency sparsity?

  2. Under Assumption B.1 (δℓ⊥u^ℓ−1∣zℓ\delta_\ell \perp \hat{u}_{\ell-1} \mid z_\ell), the unconditional covariance is Cov(δ,u^)=E[Cov(δ,u^∣z)]+Cov(E[δ∣z],E[u^∣z])\mathrm{Cov}(\delta,\hat{u}) = \mathbb{E}[\mathrm{Cov}(\delta,\hat{u}\mid z)] + \mathrm{Cov}(\mathbb{E}[\delta\mid z], \mathbb{E}[\hat{u}\mid z]). The first term vanishes, but the second does not in general because both conditional means depend on zz. What architectural constraint makes Cov(E[δ∣z],E[u^∣z])=0\mathrm{Cov}(\mathbb{E}[\delta\mid z], \mathbb{E}[\hat{u}\mid z]) = 0? If there is none, Eqs. (43) and (45) are not correct and Theorem 4.2 does not follow.

  3. Can you fix the derivation of Eq (31) from the bound (29)?

  4. Can you provide error bars / ±\pm st. deviations for the relevant tables in the main body?

4 comments

Thank you for the detailed review.

Q1. Related work and residual gating

Reply We misattributed MFPC-Net to Liu and Wang (2019) and will cite Wang and Lin [1]. Their decoder takes a coarse value at each query and uses Lagrange multipliers to constrain the fine prediction. MF-PINP learns successive corrections from separate context sets, allowing fine queries on a different grid without a coarse observation at each query.

Thank you for pointing us to MFR-PINP [2]. After reading it, we will replace the broad claim at lines 129–130 with “MFPC-Net combines multi-fidelity neural processes with physics constraints. MFR-PINP uses physics-based priors in residual neural processes for state estimation. MF-PINP uses coarse PDE residuals to guide attention over coarse observations and their fusion with fine representations.” We will credit GP-HM’s log-domain weights and Jeffreys-prior sparsity [3].

To isolate the gate’s residual input, both variants retain the gate and Eq. (12). We retrained with matched splits, seeds, and budgets, masking only the mean PDE residual in Eq. (13) in the ablation. The full model reduces mean error by 56.5%. We will add this comparison to the revision.

Gate ablation. Relative L2L^2 error (mean ± SD, 5 seeds)

BenchmarkGate residual maskedFull MF-PINP
1D Allen–Cahn u1u_11.43e−5 ± 2.32e−66.22e−6 ± 8.52e−7

Q2. Variance and calibration

Reply At a fixed query, let U=u^ℓ−1U=\hat u_{\ell-1} and D=δℓD=\delta_\ell. Given zℓz_\ell, DD is deterministic. The law of total covariance gives Cov⁡(U,D)=Cov⁡(E[U∣zℓ],D)\operatorname{Cov}(U,D)=\operatorname{Cov}(\mathbb E[U\mid z_\ell],D), which can be nonzero. Expanding Var⁡(ρℓU+D)\operatorname{Var}(\rho_\ell U+D) yields

Unrolling keeps all pairwise covariance terms. Joint latent draws preserve these dependencies when estimating predictive variance. We will include the full derivation. The Berry–Esseen step requires an independent-sum representation, which is unestablished for softmax attention and nonlinear decoding. We will replace the theorem with the variance identity and report empirical coverage. Adding coarse samples leaves Nmin⁡N_{\min} unchanged when fine fidelity is the bottleneck.

Q3. Generalization and spectral approximation

Reply We used Eq. (29)'s function-class complexity to motivate a reduced parameter KL. The complexity estimate depends on spectral weights and layer norms. Parameter KL needs its own control. With network posterior N(m,σ2I)\mathcal N(m,\sigma^2I) and fixed prior N(0,σ2I)\mathcal N(0,\sigma^2I), the network KL is ∥m∥2/(2σ2)\Vert m\Vert^2/(2\sigma^2), with no general reduction to latent dimension.

For n≥8n\ge8 i.i.d. tasks, a [0,1][0,1] task loss and a data-independent prior PP, the replacement result for randomized predictors is, with probability at least 1−η1-\eta simultaneously for all QQ of finite KL [4],

The full KL retains both spectral and network contributions. For periodic u∈Hsu\in H^s, let PKP_K project onto modes of radius at most KK. Weighting the squared Fourier tail gives

With M≍KdxM\asymp K^{d_x} modes this is O(M−s/dx)O(M^{-s/d_x}), hence O(M−1/2)O(M^{-1/2}) for s>dx/2s>d_x/2. The original pointwise-tail argument gives O(M1/2−s/dx)O(M^{1/2-s/d_x}). The learned dictionary requires a separate approximation argument.

Q4. Numerical corrections

Reply The LSE ratio is 23.2× at σ=100\sigma=100 and 127× at σ=1\sigma=1. PICA reduces Allen–Cahn u1u_1 error by 61% relative to its ablation. The stated costs imply a 9.9× data-generation ratio. Interpolation errors are 0.35% and 0.30%, between 0.52% and 0.28%. At 64² the error is 25% above the fine endpoint and 33% below the coarse. Tables 1–2 show 8/11 wins. In Table 9, the displayed OOD/ID means give 0.45/0.26≈1.730.45/0.26\approx1.73, versus the listed 1.28. We will reconcile the ratio from the run records, correct the summaries, and define symbols, units, denominators, and aggregation in the appendix.

Q5. Run-to-run variation

Reply We evaluated both methods on the same held-out tasks in each run. The paired difference Δ\Delta is GP-HM error minus MF-PINP error. Both 95% CIs lie above zero.

Repeated runs. Relative L2L^2 error (mean ± SD, 5 paired runs)

BenchmarkGP-HMMF-PINPMean Δ\Delta (95% CI)
1D Poisson u1u_18.42e−7 ± 1.58e−76.85e−7 ± 8.19e−81.57e−7 [4.3e−8, 2.71e−7]
1D Allen–Cahn u2u_28.95e−6 ± 1.45e−65.86e−6 ± 8.67e−73.09e−6 [2.177e−6, 4.003e−6]

[1] Wang & Lin (2020), MFPC-Net, arXiv:2010.01378. [2] Hunter & Enyioha (2025), MFR-PINP, arXiv:2511.08231. [3] Fang et al. (2024), GP-HM, ICLR. [4] Maurer (2004), A Note on the PAC Bayesian Theorem, arXiv:cs/0411099.

I thank the authors for the detailed rebuttal, which addressed some of my concerns.

I appreciate the corrected attributions and the additional ablation regarding the residual gating. I also like that the new table (Q5) reports std. deviations over runs, but it only shows two benchmarks.

However, I am not convinced by the responses regarding my critique of the theoretical contributions. In particular, both Theorems 4.1 and 4.2 do not hold as stated in the submitted paper and the proposed corrections / modifications do not restore the promised guarantees.

Hence, a substantial amount of my initial criticism is still valid after the rebuttal and the reasons for rejection outweigh the reasons for acceptance. I will keep my score.

Thank you for recognizing the corrected attributions and additional experiments. We address the remaining concerns with explicit theoretical statements and proof steps.

We appreciate your follow-up and the time you have devoted to our paper. Below, we provide further proof details and experimental evidence for your consideration. We respectfully ask you to reconsider your assessment. Thank you for reading this follow-up.

Q1. Generalization: complete parameter complexity

Reply For a sample SS of n≥8n\ge8 i.i.d. PDE tasks and a task loss in [0,1][0,1], let P=PsPwP=P_sP_w be a proper, data-independent prior on spectral and network parameters, and Q=QsQw∣sQ=Q_sQ_{w\mid s} a posterior with finite KL. Write R(Q),R^(Q)R(Q),\widehat R(Q) for posterior-averaged population and empirical risks. The chain rule gives

K:=KL(Q∥P)=KL(Qs∥Ps)+EQsKL(Qw∣s∥Pw).\mathcal K:=\mathrm{KL}(Q\Vert P)=\mathrm{KL}(Q_s\Vert P_s)+\mathbb E_{Q_s}\mathrm{KL}(Q_{w\mid s}\Vert P_w).

For fixed parameters ϑ\vartheta, the exponential-moment bound [1] is ESexp⁡[n kl(R^ϑ∥Rϑ)]≤2n\mathbb E_S\exp[n\,\mathrm{kl}(\widehat R_\vartheta\Vert R_\vartheta)]\le2\sqrt n, where kl\mathrm{kl} denotes binary KL. Integrating over PP, Markov, change of measure, and joint convexity yield, with probability ≥1−η\ge1-\eta, simultaneously for all such QQ,

n kl(R^(Q)∥R(Q))≤K+log⁡(2n/η).n\,\mathrm{kl}(\widehat R(Q)\Vert R(Q))\le\mathcal K+\log(2\sqrt n/\eta).

Pinsker's inequality therefore gives

R(Q)≤R^(Q)+K+log⁡(2n/η)2n.R(Q)\le\widehat R(Q)+\sqrt{\frac{\mathcal K+\log(2\sqrt n/\eta)}{2n}}.

The network cost remains: for Qw∣s=N(μw,σ2I)Q_{w\mid s}=\mathcal N(\mu_w,\sigma^2I) and Pw=N(0,σ2I)P_w=\mathcal N(0,\sigma^2I) it is ∥μw∥2/(2σ2)\Vert\mu_w\Vert^2/(2\sigma^2). This does not infer parameter KL from Rademacher complexity. For periodic u∈Hsu\in H^s, s>dx/2s>d_x/2, and Fourier projection PKP_K onto modes of radius ≤K\le K, Parseval gives

∥u−PKu∥22≤(1+K2)−s∥u∥Hs2.\Vert u-P_Ku\Vert_2^2\le(1+K^2)^{-s}\Vert u\Vert_{H^s}^2.

With M≍KdxM\asymp K^{d_x} Fourier modes, this is O(M−s/dx)O(M^{-s/d_x}) in norm. For a learned approximation vMv_M, the additional term ∥PKu−vM∥2\Vert P_Ku-v_M\Vert_2 remains explicit; the Fourier rate alone does not bound it. The risk bound above applies to the finite randomized model.

Q2. Dependence-aware variance and coverage

Reply Fix parameters, query, and contexts; assume finite second moments. Set X=(u^0,δ1,…,δL−1)⊤X=(\hat u_0,\delta_1,\ldots,\delta_{L-1})^\top and aj=∏k=j+1L−1ρka_j=\prod_{k=j+1}^{L-1}\rho_k for j=0,…,L−1j=0,\ldots,L-1 (empty product =1=1). Unrolling the decoder gives

u^L−1=a⊤X,Var⁡(u^L−1)=a⊤Cov⁡(X)a.\hat u_{L-1}=a^\top X,\qquad \operatorname{Var}(\hat u_{L-1})=a^\top\operatorname{Cov}(X)a.

This retains all covariances induced by shared latents. For calibration, let F,H,GF,H,G be the true target YY's conditional CDF, model predictive CDF, and Gaussian CDF with the model's exact mean and positive variance. For the Gaussian central interval Iα=[r,t]I_\alpha=[r,t], G(t)−G(r)=1−αG(t)-G(r)=1-\alpha. Comparing endpoints gives

∣Pr⁡(Y∈Iα)−(1−α)∣≤2∥F−H∥∞+2∥H−G∥∞.|\Pr(Y\in I_\alpha)-(1-\alpha)|\le2\Vert F-H\Vert_\infty+2\Vert H-G\Vert_\infty.

The two terms measure predictive-distribution and Gaussian-approximation error; neither is assumed small. For a fixed model and interval rule, use mm independent held-out tasks. Let Ai∈[0,1]A_i\in[0,1] be the covered fraction of target points in task ii, c^=m−1∑iAi\widehat c=m^{-1}\sum_iA_i, and c=EAic=\mathbb E A_i. Hoeffding's inequality [2] gives, with probability at least 1−η1-\eta,

∣c−(1−α)∣≤∣c^−(1−α)∣+log⁡(2/η)2m.|c-(1-\alpha)|\le|\widehat c-(1-\alpha)|+\sqrt{\frac{\log(2/\eta)}{2m}}.

Within-task grid dependence is allowed. Here mm counts independent evaluation tasks, not context points or seeds. These results replace the original dimension-only complexity and Nmin⁡−1/2N_{\min}^{-1/2} calibration claims; we will align the abstract and conclusions accordingly.

Q3. Methodological value and broader repeated-run evidence

Reply MF-PINP combines residual-guided cross-fidelity attention and gating, fidelity-adaptive spectral features, and hierarchical corrections to transfer unreliable coarse information. The five-seed ablation isolates the gate's residual input and shows a 56.5% reduction in mean error on Allen-Cahn u1u_1, with attention and the gate otherwise retained. The NS comparison below extends the repeated-run evaluation to a third PDE family.

We evaluated the three-fidelity 2D vorticity NS setting in Table 11 at Re=500\mathrm{Re}=500 using five paired runs with matched splits and data budgets.

MethodRelative L2L^2 error, mean ± SDCoverage (%), mean ± SDMean paired error difference
GP-HM5.32e-2 ± 6.15e-389.3 ± 1.82.85e-2
MFRNP3.17e-2 ± 3.02e-386.7 ± 2.27.0e-3
MF-PINP2.47e-2 ± 4.86e-391.0 ± 1.1—

Differences are baseline minus MF-PINP; coverage is at nominal 95%. We respectfully ask you to reconsider the assessment in light of the precise theoretical scope and the demonstrated value of residual-informed fusion.

References

[1] Maurer. A Note on the PAC Bayesian Theorem. [arXiv 2004].

[2] Hoeffding. Probability Inequalities for Sums of Bounded Random Variables. [JASA 1963].

I thank the authors for the follow-up and clarifications, but my criticism still stands: Both Theorem 4.1 and 4.2 are incorrect, and the proposed replacements by the authors claim and prove different (and weaker) statements, that do not support the original generalization and calibration framing. I will keep my score.

Reviewer_VSWZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

estimating neural based surrogates for numerical solutions of PDEs is an important, timely and on trend active area of invetstigtaion. The authors address the challenging problem of operator estimation with limited data (numerical solutions) at the levels of fidelity required to deliver usabale surrogates. A multi-fidelty approach which utilises Kennedy and OHagan framework to correct errors from varying levels of resolution as well as provide some UQ. The theoretical analysis I cannot vouch for. The experiemntal work looks at linear diffusions, nonlinear equations and then fluid flow problems. the emprical results are mixed with no reported SOTA - but the value of the work is the insights it provides - e.g. doesnt perform as well on smooth problems such a s diffussions.

Critical strengths
  1. It addresses a very challenging problem that lies at the heart of practical operator learning and this is a strength
  2. There is methodology and theory (which I have not checked) as well as some (limited but useful) emirical evaluation
Critical weaknesses
  1. The work is not complete - and it may be that the literature contains more comprehensive studies - nevertheless insight is imprtant.
  2. The empirical evaluation was limited - the whole area is nopw looking at models of complex physical phenomena - shocks in turbulent flow - complex geometries etc
Questions for authors

I have no questions for the authors

3 comments

Thank you for your constructive feedback and for recognizing the practical challenge of learning PDE surrogates with limited high-fidelity data. We address your two concerns below.

Q1. Scope of the contribution and relationship to prior work (Weakness 1)

Reply We agree that broader benchmark studies provide important context for our results. Our study asks a focused question: when high-fidelity simulations are scarce, how can a surrogate use coarse simulations whose accuracy varies across locations and spatial scales? MF-PINP builds on the Kennedy–O’Hagan correction framework and shares the idea of learning cross-fidelity residuals with MFRNP [1]. Our contribution is to learn spectral features adapted to each fidelity and use spatial proximity and PDE residuals to guide how coarse information is transferred. In the 2D Poisson experiment with only five high-fidelity simulations, the full model reduces relative L2L^2 error from 6.93×10−46.93\times10^{-4} to 2.47×10−42.47\times10^{-4} compared with independently modeling each fidelity (Table 6).

The gains vary across problems. GP-HM achieves lower error on the mixed-frequency 1D Poisson case u2u_2 (Table 1). On the diffusion extrapolation benchmark, MF-PINP with physics achieves 0.45%0.45\% relative error compared with 0.52%0.52\% for GP-HM with physics (Table 9). We will clarify this variation in the main text and state which comparisons support our conclusions about data efficiency.

Q2. Evaluation on shocks, turbulent flows, and complex geometries (Weakness 2)

Reply We agree that shock-containing turbulent flows and complex geometries would provide more demanding tests. Appendix C.5 reports two experiments that address parts of this concern: viscous Burgers at ν=0.01\nu=0.01, which tests a steep, smooth front, and 2D vorticity-form Navier–Stokes at Re=500\mathrm{Re}=500, which tests transfer across three grid resolutions. In the Navier–Stokes experiment, MF-PINP reduces relative L2L^2 error from 3.15%3.15\% to 2.47%2.47\% compared with MFRNP, and empirical coverage at the nominal 95%95\% level increases from 86.7%86.7\% to 92.1%92.1\%.

Table 1. Relative L2L^2 errors on the flow benchmarks (manuscript Tables 10–11)

BenchmarkFNOMFRNPMF-PINP (ours)
1D Burgers, ν=0.01\nu=0.018.74×10−48.74\times10^{-4}1.82×10−31.82\times10^{-3}3.47×10−43.47\times10^{-4}
2D Navier–Stokes, Re=500\mathrm{Re}=5004.12×10−24.12\times10^{-2}3.15×10−23.15\times10^{-2}2.47×10−22.47\times10^{-2}

We will move these comparisons into the main experimental discussion and state explicitly that shock-containing turbulent flows and irregular geometries remain open tests. Compressible flow cases from broader suites such as PDEBench [2] would help assess how the model handles shocks. For irregular domains, Appendix A.5 identifies geometry-aware spectral bases, such as mesh Laplacian eigenfunctions, as a possible extension.

References

[1] Niu et al. Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling. [ICML 2024].

[2] Takamoto et al. PDEBench: An Extensive Benchmark for Scientific Machine Learning. [NeurIPS 2022].

Reviewer_VSWZOpenReview ↗

response

Thank you for your comments and clarifications. I will retain the score as marked.

Thank you for taking the time to read our rebuttal and follow up. We appreciate your recognition of the practical challenge of learning PDE surrogates with limited high-fidelity data.

Your concern about the scope of the empirical evaluation prompted us to isolate the effect of residual gating. On the 1D Allen–Cahn benchmark, we kept the attention mechanism and data budget fixed and varied only whether the fusion gate received the coarse prediction’s PDE residual. Across five seeds, providing the residual reduced mean relative L2L^2 error by 56.5%, from 1.43×10−51.43\times10^{-5} to 6.22×10−66.22\times10^{-6}.

This experiment shows how the governing equation can guide the transfer of coarse information when fine observations are limited. It complements the three-fidelity Navier–Stokes comparison in our initial response, which evaluated the full model on a nonlinear flow problem.

In light of this additional evidence, we would be grateful if you would reconsider your overall rating. Thank you again for your time and careful review.

Comments on the paper

1 comment
Area_Chair_SCMLOpenReview ↗

Please engage in the discussion

The authors have responded to the raised questions. Reviewers please read and engage in the discussion. Thanks!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.