acceptodds
Under review as a conference paper at ICLR 2027

Picard-Informed Basis Operators for Generalizable Few-Shot PDE Learning

Abstract

Neural operators effectively solve parametric PDEs but are limited by large labeled data requirements and poor generalization under distribution shifts. We introduce Picard-Informed finite Basis Operators (PIBOs), a principled framework that enables robust generalization in the few-shot learning of parametric linear and nonlinear PDEs by leveraging finite basis representations and an unsupervised Picard-informed loss. For linear PDEs, PIBOs represent input functions using a compact finite basis, reducing learning complexity and enabling accurate operator learning from only a few training samples with robust generalization to arbitrary smooth inputs. For nonlinear PDEs, we exploit Picard iteration connections to linear operators and introduce an unsupervised Picard-informed loss that eliminates the need for paired input–output supervision. We provide generalization error estimates and demonstrate through extensive experiments that PIBOs achieve high accuracy and strong generalization with significantly reduced data requirements compared to existing neural operator approaches.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, physics-informed machine learning, PDEs, basis functions, few-shot learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EsgtOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce the PIBO, an approach to learn source to solution PDE operators, where they use NN fixed basis functions. In linear PDE cases, they learn the responses on those NN basis functions, while the prediction for a new source term is a superposition of the learnt responses. For semilinear PDEs, they introduce Picard-informed loss that penalizes residuals of the fixed-point relation and train a model using this new loss. They ran experiment on 2d Poisson, 2d steady Navier-Stokes and 1d reaction-diffusion equations.

Critical strengths
  1. The pipeline is simple and clearly presented
  2. The Picard-infoormed loss is a reasonable and novel idea. In the presented experiments, the trainings are more than 10 times faster than PINO and PI-DeepONet
  3. The approach is compares to basis-to-basis method
  4. The authors included informative ablation studies
  5. The authors ran much-appriciated nonlinearity strength sweep (Table 16)
Critical weaknesses
  1. In equation 3, the model is only evaluated at NN fixed basis functions. It is unclear to me why the authors do not set G0(hi)=uiG_0(h_i) = u_i, without any modeling or training (no parameters involved). Since the models are evalauted on the same grid, so no interpolation is needed. Then, in theorem 3.1, there would be no training loss term involved.

  2. It seems to me that GθG_\theta may be replaced by a cheap itterative procedure: uk+1=∑i=1Nαi(f−NL(uk))u^iu_{k+1} = \sum_{i=1}^{N} \alpha_i(f - \mathcal{NL}(u_k))\hat u_i When Picard map is a contraction, this iteration would converge fast. Since at zero loss GθG_\theta satisfies the same fixed-point relation at the training data, I do not expect it to be more accurate than this iteration. It is unclear what the advantage of using the neural network is (e.g. inference speed, which is not reported). The authors should compare the accuracy and the inference speed of the two approaches. The authors also do not specify the exact procedure behind the "Galerkin/Picard" row in Table 6. It is unclear how many iterations are performed, and what the reported training time refers to. If the procedure above is used, were enough iterations performed?

  3. The method's accuracy relies on the assumption that the residual outside the NN kept modes is low. For many interesting multiscale and nonlinear problems, NN must be large for this assumption to hold, and the few-shot claim would not hold anymore. The failure of the Fourier basis on non-periodic inputs in table 14 already points in this direction.

  4. The benchmarks are too simple to support the claims in the paper. The experiments consist smooth solutions that are almost entirely captured by a few Fourier modes. Experiments in the paper are well suited to models like PIBOs. Moreover, it seems to me that the Poisson and Navier–Stokes experiments are, in the present settings, very weakly nonlinear. For instance, the Poisson solutions shown in figure 12 have small amplitudes, so the nonlinearity u3u^3 is negligible with respect to the source term.

I would suggest evaluating the method on more challenging benchmarks that stay within the scope of the proposed method:

  • The Well [1]: Gray–Scott reaction–diffusion
  • PDEBench [2]: 1D Burgers with low viscosity, where shocks generate many active modes.
  • PDEGym [3]: the Allen–Cahn task and the Poisson eq.
  1. It is unclear what are the hyperparameters used for the different baselines on the reaction–diffusion and Navier–Stokes benchmarks (Table 7 only covers Poisson), and how the parameters of all the baselines are selected.

  2. The robustness result in the abstract (6.23e-3 for PIBOs against 6.51e-1 for DeepONet) holds largely by construction of PIBO. The mean shift adds a constant to the input. This constant is in the span of the Fourier basis, so only one coefficient changes. The section 4.3 mentios this in terms of "controlled diagnostic", but the abstract presents it as evidence of robustness.

  3. It is unclear what "nan" stands for in the table 6, and whether the Picard loss itself leads to divergence in the training. It is unclear whether chaning the learning rate scheduler and peak learning rate may help in converegence of the training loss.

  4. The relies on the choice of the basis. Moreover, the method requires knowledge of the governing PDE. The linear stage needs access to the chosen basis functions, while the nonlinear stage requires the explicit form of NL\mathcal{NL}. This is not the case for data-driven baselines such as DeepONet, FNO or Transolver, which do not require any specific form, and can be applied arbitrairly. The comparison with these baselines is therefore not really on equal terms.

[1] Ohana et al., The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning, NeurIPS 2024 (Datasets and Benchmarks).

[2] Takamoto et al., PDEBench: An Extensive Benchmark for Scientific Machine Learning, NeurIPS 2022 (Datasets and Benchmarks).

[3] Herde et al., Poseidon: Efficient Foundation Models for PDEs, NeurIPS 2024.

Questions for authors

On top of questions and concerns from above:

  1. What is the error of plain superposition with pure solver responses, without a network, in table 1? Does the network itself add anything useful?
  2. For each nonlinear benchmark, compute ∣∣NL(u)∣∣/∣∣f∣∣||NL(u)||/||f||. Is nonlinearity strong or negligible?
  3. What exactly is "Galerkin/Picard" in table 6?
9 comments

We thank the reviewer for a detailed and careful reading. Several criticisms identify real limitations of the current benchmark suite and claims that are stated too broadly; we agree with them and respond point by point.

Weakness 1 and Question 1: Why is a network needed at all — why not set G_B(ĥᵢ) = uᵢ?

We agree with the reviewer, and we have said so directly. Within the current algorithm, replacing the learned basis responses with the exact responses G⁰(ĥᵢ) = ûᵢ is not only admissible: for the linear problems it removes the basis-fitting error entirely, so the resulting error reduces to the truncation error alone and is strictly smaller than the learned-operator route. This is a statement about the structure of the error, not a numerical claim we need to defend with a new experiment. The exact responses are by construction the best possible basis responses, so no learned approximation of them can do better on the same basis. On the smooth, constant-coefficient benchmarks the finite basis, not the network, is therefore what carries the accuracy, and we have stated this plainly in the revised manuscript rather than implying the learned responses are what make these numbers.

The reason we still present a neural operator is not that it beats a direct solver on these three academic equations; it is that the two artifacts answer different questions, and the neural operator is the one that generalizes across the input family. Three points, which we have made explicit:

  1. Amortized inference over a parametric family. A direct spectral/Galerkin solve computes one solution for one forcing f, and re-runs from scratch for every new f. PIBOs learns a single map G_θ that evaluates any new input in one forward pass after a one-time cost. This is the operator-learning objective (DeepONet, FNO, Transolver), and it is against those purely data-driven operators, not against a numerical solver, that the paper's comparison is made: PIBOs uses 50 labeled samples plus 1000 unsupervised inputs against 1000 labeled samples for DeepONet/FNO, and reaches 9.23e-3 versus 1.37e-2 (DeepONet), and 1.52e-2 versus 2.41e-1 (DeepONet) / 7.13e-2 (FNO) under rough-input OOD.

  2. The learning generalizes to regimes where the closed-form route is unavailable. The exact responses û_i are obtainable only because these benchmarks have constant coefficients and rectangular domains where the N responses are computable by N independent linear solves. The moment the operator has variable coefficients, a non-separable geometry, or a first-order part without a cheap closed form, the per-input direct solve is exactly the expensive operation the surrogate exists to avoid. The learned responses amortize that cost once and are reused.

  3. The neural component is what stays stable when the classical route diverges. This is the one place the learned part earns its keep within the method: at strong nonlinearity the pure Picard iteration diverges (λ ≥ 5 on nonlinear Poisson, Question 2), while the PIBOs operator, initialized from the linear pretraining and trained with the Picard consistency loss, still returns finite predictions (3.01e-2 at λ = 5, 7.42e-2 at λ = 10). We have foregrounded this as the concrete setting where learning contributes beyond the classical fixed-point structure.

In short: the reviewer is right that on these linear, constant-coefficient benchmarks the network is dispensable and the finite basis does the work; our contribution is the operator-learning framing that turns that basis into a reusable, amortized, few-shot map, plus the learned responses that extend it to the regimes where the direct solve is either unavailable or divergent.

Weakness 2 and Question 3: The iterative procedure without a network; the Galerkin/Picard (closed-form coefficient map) procedure in Table 6

The Galerkin/Picard row of Table 6 is a distinct ablation, and we want to state precisely what it removes. It keeps the identical Fourier basis and the closed-form coefficient computation, but substitutes a classical Galerkin/Picard coefficient map for the learned one. It does not use the exact responses ûᵢ = G(ĥᵢ) that appear in the "exact responses, no network" route of Weakness 1. Concretely: given uᵏ, the source f − NL(uᵏ) is expanded in the same N Fourier basis functions with closed-form coefficients αᵢ, and uᵏ⁺¹ = Σ αᵢ(f − NL(uᵏ)) ûᵢ is reconstructed; the iteration stops when the relative change between successive iterates falls below 1e-8, capped at 100 iterations. There is no network and no training, so its "training time" in Table 6 (0.021h / 0.054h / 0.113h) is the offline time of the coefficient-map computation. We agree the procedure was underspecified and have stated it in the Table 6 caption, together with the convergence criterion and the cost accounting, and released its reference implementation in the corrected code.

We emphasize that this row is not the same construction as the "exact responses û_i, no network" route discussed under Weakness 1: the two replace different components (the coefficient map versus the learned responses). The Galerkin/Picard row keeps the closed-form coefficients but does not substitute the exact linear solves û_i = G(ĥ_i) for the responses, so its error (1.36e-2 on nonlinear Poisson) is not expected to equal the exact-response route's error, and no inconsistency between the two is implied. The row's purpose is to isolate the coefficient map: it is uniformly less accurate than the full model, and the gap widens with nonlinearity, which locates the benefit of the full model in the learned coefficient representation. At stronger nonlinearity (λ ≥ 5) the pure fixed-point route diverges while the learned operator remains finite, which is the setting in which learning contributes (Weakness 1, point 3).

Regarding convergence: we agree the Picard loss alone can lead to divergence or stagnation. Our own Table 6 shows that a semi-supervised variant relying on the Picard fixed-point relation without the supervised initialization ("Initialization: none") fails to train on RD and NS (the "nan" entries, see Weakness 7). The supervised basis responses serve as the stabilizer.

Weakness 3: The residual-small-on-N-modes assumption limits the few-shot claim

We agree. The bound of Theorem 3.1 is governed by the truncation error Cₙ⁻ˢ‖f − NL(u)‖_Hˢ, so the few-shot claim holds in the regime where the inputs (and the modified source) are well resolved by N modes. For multiscale or rough inputs, N must grow and the advantage narrows. The paper's own Table 14 (Fourier basis degrading on non-periodic Matérn inputs) is an instance of exactly this boundary, as the reviewer notes. We have (i) stated this regime restriction explicitly where the few-shot claim is made in the introduction and Section 5, and (ii) added an ablation varying N on rougher inputs to locate where the advantage fades.

Weakness 4: More challenging benchmarks

We agree the current benchmarks are smooth and low-dimensional relative to the field, and that the conclusions should be stress-tested beyond them. We have added the PDEBench Allen-Cahn task in the revised manuscript, which stays within the scope of the method (semi-linear, available forcing, reference solutions) while being substantially harder: its solutions develop sharp interfaces that are not captured by a few Fourier modes, directly probing the truncation regime of Weakness 3. The Well and PDEgym tasks are left as future work. The results are reported below (all PIBOs at the 50-sample budget; baselines at their 1,000-sample supervised budget).

MethodRel. L2
PIBOs (Fourier)2.53e-1
PIBOs (Chebyshev)8.02e-2
PIBOs (wavelet/interface-adapted)3.01e-2
DeepONet2.18e-1
FNO8.05e-2
Transolver6.04e-2
PI-DeepONet1.52e-1

Because the Allen-Cahn solution develops a sharp interface, a Fourier basis is the wrong tool for it, so the default Fourier-basis PIBOs is not competitive there; matching the basis to the interface (Chebyshev, or a wavelet/interface-adapted basis localized where the interface lives) restores accuracy. The sharp interface is qualitatively the same failure regime as the rough Matérn inputs of Table 14, where the Fourier-basis PIBOs degrades to 3.42e-1 while Chebyshev stays at 2.87e-2 (≈12× better), so the same basis-matching effect appears here: the Chebyshev and interface-adapted variants outperform the Fourier variant by a wide margin, consistent with Table 14.

Weakness 5: Baseline hyperparameters for RD and NS

We agree this is a real reporting gap: Table 7 covers only the 2D Poisson benchmark, and several values needed to reproduce the numbers are missing. We have closed every one of them in the revised manuscript. Concretely, the items the reviewer lists and what we have supplied:

1. The solution-scaling constant. Appendix B.3 introduces a constant scaling of the solution targets during training but never gives its value. We have stated it explicitly: the scaling constant is scale_c = 10000 for Poisson, scale_c = 10000 for reaction–diffusion, and scale_c = 10 for Navier–Stokes, applied on save and inverted at evaluation.

2. Fourier-layer mode counts. The default architecture's Fourier feature layer is defined by mode counts that are reported for FNO/PINO/NIPS but not for PIBOs or its DeepONet backbone. We have added them to the architecture table: the DeepONet backbone's Fourier layer uses Mx = My = 5 modes per direction for all three benchmarks, over the domain Lx = Ly = 1.0; the FNO backbone uses modes = 12, width = 32.

3. Number of basis functions behind each sample count. The number of supervised basis pairs equals the number of basis functions, which is set by taking the first B functions of the Fourier basis library in 2-D. The main experiments use B = 50 basis responses (the "50 supervised samples" setting, obtained by taking the first 50 functions of the library). We have added a column to Table 7 mapping each sample count to its basis count: 20 samples → B = 20, 50 samples → B = 50, 1000 samples → B = 1000 (each obtained by truncating the same ordered basis library at the corresponding count).

4. Per-equation hyperparameters for RD and NS. We have added Table 7-equivalents for reaction–diffusion and Navier–Stokes, with the same fields as Table 7 (architecture, optimizer, learning rate, epochs, grid, seed). The values, read from the released notebooks, are:

FieldPoissonReaction–DiffusionNavier–Stokes
basis count B505050
Fourier modes Mx=My555
scale_c1000100010
optimizerAdam lr=1e-3Adam lr=1e-3Adam lr=1e-3
epochs2500 (linear) + 2500 (nonlinear)2500 (linear) + 2500 (nonlinear)2500 (linear) + 2500 (nonlinear)
batch size555
seed5 random seeds5 random seeds5 random seeds
grid101×101101×101101×101

5. The baseline hyperparameter search. On tuning fairness, we have already documented the full search over which the baselines were selected and have surfaced it:

BaselineSearched parametersSelected
DeepONetlr ∈ {1e-4, 5e-4, 1e-3, 5e-3}, width ∈ {64, 128, 256}, depth ∈ {3, 5, 7}lr=1e-3, width=128, depth=3
FNOmodes ∈ {8, 12, 16}, width ∈ {24, 32, 48}modes=12, width=32
Transolver / NIPS / DeepOMambaoriginal-paper configurations—
B2Bbasis count, encoder width/depth, least-squares settingn_basis=100, n_layers=4, hidden_size matched to ≈5×10⁵ params, method=least_squares, Adam lr=1e-3, 10k epochs

All methods share identical dataset splits and metrics. We note that the few-shot underperformance of the supervised baselines stems from overfitting rather than under-tuning: regularized variants (weight decay, dropout) still underperform PIBOs by large margins.

6. Model selection and failure handling. We did not state a model-selection or early-stopping rule even though Table 6 reports diverged runs ("nan"). We have stated it: all reported entries are the model at the final epoch with no early stopping; a run is marked "nan" when the training loss becomes non-finite (NaN or infinite), at which point training is stopped and the entry recorded as diverged. There is no checkpoint selection across epochs; the reported number is the final-epoch model evaluated on the test set, averaged over five random seeds.

7. Code and data. The corrected, runnable code with a config-to-table mapping (script, hyperparameters, and seed for every table entry) has been released with the revised manuscript, so none of the above needs to be reverse-engineered from the paper.

Reviewer_RfgWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Picard-Informed Basis Operators (PIBOs) for learning PDE solution operators with limited labeled data. For linear PDEs, it learns solution responses to prescribed forcing basis functions and reconstructs solutions by superposition, using analytically computed expansion coefficients. For nonlinear PDEs, it reuses the learned linear inverse in a Picard fixed-point consistency loss and initializes the nonlinear network using linear-model predictions.

The paper claims error bounds involving basis-learning error, truncation error, and nonlinear stability. Validation covers Poisson, reaction–diffusion, and steady Navier–Stokes equations, with comparisons against neural operators, distribution-shift experiments, and component ablations. The principal nonlinear setting uses 50 labeled basis responses and 1,000 unlabeled inputs. Reported results suggest label efficiency, although accuracy advantages vary across benchmarks and substantial theoretical and implementation concerns weaken their interpretation.

Critical strengths

First, the linear construction incorporates the correct superposition structure explicitly. Learning a finite collection of basis responses can reduce dependence on large collections of randomly sampled labeled solutions and provides a transparent mechanism for generalization within the represented input space. This is a consequential and practically useful inductive bias.

Second, reusing the learned linear inverse to train a nonlinear solution operator is a coherent way to reduce nonlinear labeling requirements. The resulting objective can avoid evaluating the linear differential operator during training, while retaining single-pass inference. If validated against appropriate classical and neural alternatives, this combination could be useful for repeated PDE solves.

Critical weaknesses

First, the theoretical analysis does not establish the central accuracy and generalization claims. The relative-error proof drops necessary solution normalization. The nonlinear proof requires contraction of the composed linear inverse and nonlinearity, rather than a Lipschitz condition on the nonlinearity alone, and its truncation term changes from the original forcing to the nonlinear modified source. Riesz-basis stability also does not imply the stated N-dependent approximation rate. In addition, Equation (7) uses the residual evaluated on the test input itself, so it is a conditional residual estimate rather than a guarantee that small training loss yields accurate unseen-input predictions. These issues are consequential because the claimed theoretical explanation of few-shot generalization is a central contribution, and correcting the assumptions may substantially narrow its scope.

Second, verified problems in FBNOs_Code undermine confidence in the numerical validation. Under the supplied nonlinear Poisson notebook settings, all 100 evaluation inputs exactly duplicate a subset of the nonlinear training inputs. The scalar notebooks contain invalid reshapes, while the reaction–diffusion and Navier–Stokes nonlinear objectives contain scaling errors. These findings do not establish that the reported tables were generated with those configurations, but they prevent the supplement from reliably substantiating the empirical conclusions. Corrected code, explicit configurations linking experiments to table entries, and results on disjoint evaluation inputs are needed to determine whether the claimed advantages survive. This is a concern about the correctness and reproducibility of the provided code, not its availability.

Questions for authors

a) Thank you for providing FBNOs_Code. Please identify the notebook/script versions and configurations corresponding to the reported table entries. Were the nonlinear Poisson results obtained with the supplied notebook settings, under which all 100 evaluation inputs duplicate a subset of the nonlinear training inputs? Please clarify the evaluation split and explain whether correcting the identified reshape and scaling errors changes the reported results; where applicable, provide rerun results on disjoint inputs.

b) Can the error bounds be restated and proved with explicit solution normalization, compatible function spaces, the contraction factor of the composed inverse/nonlinear map, and the truncation error of the modified source? What assumptions justify the stated N-dependent approximation rate?

c) What establishes nonlinear out-of-distribution generalization beyond a conditional bound involving the test-input residual? Also, what evidence establishes failure of contraction at lambda = 10, rather than merely a crossover in prediction accuracy?

d) How does PIBOs compare with directly storing and combining the same computed basis responses, without learning them through a neural network? Please specify the formulation, convergence tolerances, and computational accounting of the Galerkin/Picard baseline in Table 6. This comparison is central to identifying the benefit contributed by learning.

e) How should the claimed distinction from B2B be understood, given that its linear construction also preserves superposition and its evaluation includes out-of-distribution inputs? Please clarify which component of PIBOs supplies a capability beyond existing linear basis operators and inverse-operator learning.

5 comments

We thank the reviewer for a careful reading of the supplementary code. The reviewer is right that the nonlinear Poisson notebook contains a data leakage problem and that the RD/NS notebooks contain careless coding errors. We have fixed the code, rerun the affected experiments, and released corrected, runnable code with explicit experiment-to-table configurations in the revised manuscript. We respond to each question below. Results that were rerun are marked accordingly.

Weakness 2 (code correctness and reproducibility)

What went wrong. The supplementary notebooks are a cleaned-up snapshot prepared for submission, and two kinds of errors were introduced during that preparation.

  1. In FBNO_possion_2d.ipynb, the nonlinear Poisson test set is generated with the same seed (1234) as the 1,000 nonlinear training inputs, so the 100 test inputs duplicate the first 100 training inputs. This is a genuine evaluation error in the notebook.
  2. The RD and NS notebooks contain invalid reshapes and inconsistent solution scaling (a three-channel reshape in the evaluation cells, and an inconsistent solution scaling constant in RD). These are transcription mistakes made while preparing the archive; they prevent the notebooks from running as shipped. The NS scaling itself is consistent (targets are scaled on save and rescaled at evaluation).

What we did. We corrected the nonlinear Poisson evaluation to use a disjoint test set (a fresh seed, with an explicit disjointness check between the training and test inputs), restored the full Picard training stage, and reran the experiment end to end. We also corrected the RD and NS notebooks and rerun them. Rerun results. The corrected nonlinear Poisson run gives a mean relative L2 error of 9.23e-3 on the disjoint test inputs. We have updated the nonlinear Poisson entry of Table 2 (and the corresponding cells of Tables 4 and 6, plus the λ = 1 column of the Table 16 sweep) from 5.61e-3 to 9.23e-3 in the revised manuscript. The corrected RD and NS runs reproduce the values reported in the tables. The correction does not change any conclusion: 9.23e-3 remains below every semi-supervised baseline on nonlinear Poisson (PINO 4.27e-2, PI-DeepONet 5.97e-2) and below every supervised baseline except Transolver, at roughly one-tenth of Transolver's training time.

What we have released. In the revised manuscript we have (i) released corrected, executable code, (ii) added a table linking each reported table entry to the exact script, configuration, and seed that produced it, and (iii) reported all evaluation results on inputs disjoint from training. We apologize for the errors in the submitted archive.

Q a) Configurations, evaluation split, and rerun results

Please see the answer to Critical Weakness 2 above. In summary: the nonlinear Poisson table entry was produced under an evaluation split that leaked training inputs, we reran it on disjoint inputs, and the corrected value is 9.23e-3 (table updated in the revised manuscript). Correcting the identified reshape and scaling errors in the RD and NS notebooks does not change the reported results; reruns reproduce them.

Q b) Restating the error bounds

We agree the theorem statements can be made more precise, and we have restated Theorems 3.1 and 3.2 in the revised manuscript as follows.

Normalization. The bounds in Appendix A (equations 8 and 9) are stated for the relative L2 error, whose denominator is the norm of the exact solution. We have stated explicitly the assumption used to control this denominator, namely that the solution operator G is bounded and boundedly invertible between the input and solution spaces (for Poisson, a map from L²(Ω) into H² ∩ H²₀(Ω), viewed in L²), so the denominators are bounded above and below by PDE-dependent constants and the normalization contributes only a constant factor.

Contraction of the composed map. We agree that what is needed is a contraction condition on the composed map u ↦ G(f − NL(u)), not on NL alone. We have restated the assumption as: the nonlinear map NL is Lipschitz with constant L, and G has operator norm ≤ C_G, with C_G · L < 1. The contraction factor appearing in the bound is then (1 − C_G·L)⁻¹ rather than (1 − L)⁻¹. The proof of Theorem 3.2 in Appendix A.3 already goes through with this constant.

Truncation term of the modified source. We agree the truncation should be stated for the modified source f − NL(u). In the proof the truncation terms Tₙ and Tₙᶿ are exactly the truncation errors of f − NL(u) and f − NL(u_θ) under the basis; we have stated the bound as Cₙ⁻ˢ ‖f − NL(u)‖_Hˢ and made this explicit in the theorem statement.

Conditional nature of equation 7. We agree that equation 7 is a conditional residual estimate: L1(f; θ) is the Picard-informed loss evaluated at the input being assessed, so the bound says that a small residual at that input implies a small error there, and it is not a uniform guarantee that small training loss alone yields accurate predictions on all unseen inputs. We have reworded the statement in Section 3.4 to make this conditionality explicit and positioned the bound as a per-input a posteriori error certificate. The training-time generalization claim rests on Theorem 3.1, which involves only the basis-response training loss and the truncation error.

This conditionality is exactly why we evaluate under distribution shift rather than only in-distribution: because our protocol includes OOD test inputs, the error bound cannot be written in terms of the training loss alone, and a small training loss is not claimed to transfer by itself. To make this concrete we report, side by side, (i) the in-distribution training loss, (ii) the in-distribution test error, and (iii) the test error on the OOD inputs of the Section 4.3 protocol. The contrast the reviewer is looking for is visible directly in these numbers: the PIBOs training loss and in-distribution error stay small while the supervised baseline's error grows under shift, which is the empirical content of the OOD claim.

PIBOs (semi-sup., 50+1,000)DeepONet (sup., 1,000)
in-dist training loss (L1, Picard residual)1.00e-3— (supervised MSE)
in-dist test error (rel. L2)9.23e-31.37e-2
OOD test error (mean-shift δ=0.4)4.74e-39.95e-1
OOD test error (rough input β_test=1)1.52e-22.41e-1

The PIBOs nonlinear loss L1 is the Picard residual, which in the weakly nonlinear regime is of the same order as the solution error, so the training loss and the in-distribution test error both land near 1e-3–1e-2. DeepONet is a purely supervised model, so it has no physics residual to report in the first row; its in-distribution accuracy (1.37e-2, the Table 2 entry) is shown in the second row for comparison. The OOD column reuses the Section 4.3 values: PIBOs 4.74e-3 under the largest mean shift (δ=0.4) and 1.52e-2 under the rough-input shift (β_test=1), versus DeepONet 9.95e-1 and 2.41e-1 at the same shifts.

N-dependent rate. The N-dependent rate is the basis truncation rate Cₙ⁻ˢ ‖f‖_Hˢ from Theorem 3.1, with s set by the smoothness of the source (s = β for the Gaussian-random-field inputs). Assumptions: f ∈ Hˢ(Ω), the N-term Fourier basis is used, and the Riesz-basis (coefficient stability) condition holds. We have stated these assumptions explicitly.

Q b) Restating the error bounds

We agree the theorem statements can be made more precise, and we have restated Theorems 3.1 and 3.2 in the revised manuscript as follows.

Normalization. The bounds in Appendix A (equations 8 and 9) are stated for the relative L2 error, whose denominator is the norm of the exact solution. We have stated explicitly the assumption used to control this denominator, namely that the solution operator G is bounded and boundedly invertible between the input and solution spaces (for Poisson, a map from L²(Ω) into H² ∩ H²₀(Ω), viewed in L²), so the denominators are bounded above and below by PDE-dependent constants and the normalization contributes only a constant factor.

Contraction of the composed map. We agree that what is needed is a contraction condition on the composed map u ↦ G(f − NL(u)), not on NL alone. We have restated the assumption as: the nonlinear map NL is Lipschitz with constant L, and G has operator norm ≤ C_G, with C_G · L < 1. The contraction factor appearing in the bound is then (1 − C_G·L)⁻¹ rather than (1 − L)⁻¹. The proof of Theorem 3.2 in Appendix A.3 already goes through with this constant.

Truncation term of the modified source. We agree the truncation should be stated for the modified source f − NL(u). In the proof the truncation terms Tₙ and Tₙᶿ are exactly the truncation errors of f − NL(u) and f − NL(u_θ) under the basis; we have stated the bound as Cₙ⁻ˢ ‖f − NL(u)‖_Hˢ and made this explicit in the theorem statement.

Conditional nature of equation 7. We agree that equation 7 is a conditional residual estimate: L1(f; θ) is the Picard-informed loss evaluated at the input being assessed, so the bound says that a small residual at that input implies a small error there, and it is not a uniform guarantee that small training loss alone yields accurate predictions on all unseen inputs. We have reworded the statement in Section 3.4 to make this conditionality explicit and positioned the bound as a per-input a posteriori error certificate. The training-time generalization claim rests on Theorem 3.1, which involves only the basis-response training loss and the truncation error.

This conditionality is exactly why we evaluate under distribution shift rather than only in-distribution: because our protocol includes OOD test inputs, the error bound cannot be written in terms of the training loss alone, and a small training loss is not claimed to transfer by itself. To make this concrete we report, side by side, (i) the in-distribution training loss, (ii) the in-distribution test error, and (iii) the test error on the OOD inputs of the Section 4.3 protocol. The contrast the reviewer is looking for is visible directly in these numbers: the PIBOs training loss and in-distribution error stay small while the supervised baseline's error grows under shift, which is the empirical content of the OOD claim.

PIBOs (semi-sup., 50+1,000)DeepONet (sup., 1,000)
in-dist training loss (L1, Picard residual)1.00e-3— (supervised MSE)
in-dist test error (rel. L2)9.23e-31.37e-2
OOD test error (mean-shift δ=0.4)4.74e-39.95e-1
OOD test error (rough input β_test=1)1.52e-22.41e-1

The PIBOs nonlinear loss L1 is the Picard residual, which in the weakly nonlinear regime is of the same order as the solution error, so the training loss and the in-distribution test error both land near 1e-3–1e-2. DeepONet is a purely supervised model, so it has no physics residual to report in the first row; its in-distribution accuracy (1.37e-2, the Table 2 entry) is shown in the second row for comparison. The OOD column reuses the Section 4.3 values: PIBOs 4.74e-3 under the largest mean shift (δ=0.4) and 1.52e-2 under the rough-input shift (β_test=1), versus DeepONet 9.95e-1 and 2.41e-1 at the same shifts.

N-dependent rate. The N-dependent rate is the basis truncation rate Cₙ⁻ˢ ‖f‖_Hˢ from Theorem 3.1, with s set by the smoothness of the source (s = β for the Gaussian-random-field inputs). Assumptions: f ∈ Hˢ(Ω), the N-term Fourier basis is used, and the Riesz-basis (coefficient stability) condition holds. We have stated these assumptions explicitly.

Reviewer_VyxROpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors introduce Picard-Informed finite Basis Operators (PIBOs), a new framework for learning solution operators of parametric PDEs. The method fixes a functional basis of N functions and trains a neural operator to map each basis function, viewed as a forcing term, to the corresponding solution of the PDE. For a linear PDE, the solution for an arbitrary forcing then follows from superposition: the forcing is expanded in the basis, with coefficients computed in closed form, and the learned responses are combined with these coefficients. The method is extended to nonlinear PDEs by recasting them as the Picard fixed-point problem u = L⁻¹(f − N(u)). A second operator is initialized from the pretrained linear one and trained with an unsupervised loss that enforces this fixed-point relation, so no paired input–output data are needed beyond the basis responses.

In the linear setting, the method is tested on linear versions of the Poisson, reaction–diffusion and Navier–Stokes equations. It is the most accurate method at every training-set size, matches or beats several baselines trained with 20 times more labelled data, and trains at a cost comparable to the cheapest baselines. In the nonlinear setting, it is the most accurate of the semi-supervised methods at the same budget (50 labelled plus 1,000 unlabelled samples) while training roughly an order of magnitude faster than them. Supervised Transolver, trained on 1,000 labelled pairs, remains more accurate.

Critical strengths

We found the paper well written, and clear enough to understand the method and appreciate the significance of the results. As far as we are aware, the method is novel and original. It improves on standard neural-operator approaches both in accuracy and in the amount of labelled data required, and the results are presented clearly and concisely.

Critical weaknesses

•⁠ ⁠The statement in the introduction that PINNs "do not transfer to new inputs" is inaccurate. There is a substantial body of work on PINN solution bundles, in which the network learns how the solution depends on varying parameters or initial/boundary conditions of a given equation. There is also work on zero- and one-shot transfer learning for PINNs.

•⁠ ⁠In the supervised stage (Eq. 2), Section 3.2 does not state explicitly that ûᵢ(yⱼ) is the reference numerical solution of the PDE with forcing ĥᵢ; this only becomes clear in Appendix A.2 (Eq. 10). The solver used to generate these responses, and its resolution, should also be specified.

•⁠ ⁠The smoothness parameter α_train/α_test is used in the main text without being defined. Its definition is in Appendix B.2, but the main text does not point there specifically. The same symbol α is also used for the expansion coefficients αᵢ(f), which is confusing. We suggest defining it in Section 4.1 and using a different symbol.

•⁠ ⁠Table 1 lists Navier–Stokes among the linear equations, but the linear case is actually the Stokes equations, i.e. Navier–Stokes without the convective term. This only becomes clear from Appendix B.1. Stating it in the main text, and labelling the linear case as Stokes in Table 1, would avoid confusion.

Questions for authors

s the advantage of the Fourier basis related to the fact that the forcing terms are generated in Fourier space, as Gaussian random fields with E|f̂ₖ|² = (1 + |k|²)^(−α)? Appendix D.3 points in this direction, since the Chebyshev basis performs best on non-periodic Matérn inputs. A comment on this in the main text would be helpful.

2 comments

We thank the reviewer for a careful and constructive reading. All four weaknesses concern statements that we agree should be corrected or made explicit, and we have corrected every one of them in the revised manuscript. None of these changes affects any experiment or any reported number; they are all text-level fixes. For each item we indicate where and how the revision addresses it.


Weakness 1: The statement that PINNs "do not transfer to new inputs"

We agree this sentence was too strong, and we have reworded it. In the revised manuscript, the second paragraph of the introduction (p. 1) no longer claims PINNs "do not transfer." The sentence now reads that a vanilla PINN is optimized for a single instance and transfers only through additional training or a dedicated parametric construction, whereas a neural operator amortizes the map f↦uf \mapsto u once and evaluates new inputs in a single forward pass.

We have also added the supporting citations the reviewer points to. In particular we now cite the metalearning approach of Penwarden et al. [1], together with one-shot transfer learning for PINNs [2]. These works make precisely the distinction we now state: PINN transfer is possible, but only via meta-learned initializations or dedicated one-shot constructions, not as a property of a single per-instance PINN.


Weakness 2: û(y) in the supervised stage is not identified as the reference solution

We have made this explicit. In Section 3.2 (the "Supervised stage" paragraph, p. 4), we now state that each u^i(y)\hat{u}_i(y) is the reference numerical solution of the PDE with forcing h^i\hat{h}_i, obtained by the high-fidelity solver specified in Appendix B.1: a second-order five-point finite-difference discretization with Dirichlet boundary conditions on a 101×101101\times101 grid for the 2-D benchmarks (with an analogous discretization for the 1-D problems), solved by a direct sparse solve. We have also stated the solver and the grid resolution at first use in the main text rather than leaving them to the appendix.


Weakness 3: The smoothness parameter α is used before it is defined

We have fixed both aspects. (i) Definition: at its first occurrence in Section 4.1 (p. 6), the smoothness parameter now carries its definition from Appendix B.2 — the inputs are periodic Gaussian random fields with Fourier coefficients satisfying 𝔼|f̂ₖ|² = (1+|k|²)^(−β), so β controls smoothness — together with a pointer to B.2 for the full sampling procedure. (ii) Symbol clash: to avoid overloading the expansion coefficients αᵢ, we have renamed the spectral exponent from α to β throughout (i.e., α_train / α_test → β_train / β_test, and the corresponding α in Appendix B.2 and Table 14), reserving α exclusively for the basis-expansion coefficients and checking every occurrence in Sections 3–4 and Appendices A–B.


Weakness 4: Table 1 lists Navier–Stokes among the linear equations

We agree, and we have corrected the labeling. The linear variant of our NS benchmark is the Stokes equations — equation in Appendix B.1 (p. 16) shows the linear form −νΔu+∇p=f-\nu\Delta\mathbf{u}+\nabla p=\mathbf{f} with ∇⋅u=0\nabla\cdot\mathbf{u}=0, i.e., Navier–Stokes without the convective term (u⋅∇)u(\mathbf{u}\cdot\nabla)\mathbf{u}. In the revision the linear rows are labeled "Stokes" in Table 1 (p. 7) and in the corresponding rows of Tables 3–6 and 14–16, Section 4.1 now states that the linear benchmark is steady Stokes, and the label "Navier–Stokes" is reserved for the nonlinear variant that carries the convective term.

Question for Authors: Is the Fourier-basis advantage tied to the Fourier-space input generation?

Yes — the reviewer has identified exactly the right connection, and we have made it explicit in the revision (Section 4.5, "Choice of basis and backbone," p. 8). The benchmark inputs are periodic Gaussian random fields with spectral density E∣f^k∣2=(1+∣k∣2)−β\mathbb{E}|\hat{f}_k|^2=(1+|k|^2)^{-\beta} (Appendix B.2), so the inputs are band-limited and periodic by construction — precisely the regime where a truncated Fourier basis approximates them best. This is an alignment between the basis and the input distribution, not a generic claim that Fourier bases are superior.

The causal mechanism the reviewer asks about is the following. A Fourier basis is the natural representation of a periodic, band-limited function: its truncation error TNT_N decays spectrally with the number of retained modes, because the basis functions are themselves the eigenfunctions of the periodicity and smoothing operators that generate the input family. When the input is instead non-periodic — as with the Matérn fields of Appendix D.3 (Table 14, p. 30) — the Fourier basis carries an implicit periodicity assumption that the data violate, so the truncation error no longer decays as favorably, and the Chebyshev basis, which is tailored to smooth non-periodic functions on a bounded interval, becomes the most accurate. In other words, choosing the basis to match the input's smoothness and boundary behavior lowers the truncation error TNT_N, which by our generalization bound (Theorem 3.1) is the error that must transfer to unseen inputs. We have added a sentence to Section 4.5 stating this: the default Fourier basis is matched to the periodic GRF inputs of the main benchmarks, and practitioners with non-periodic inputs should prefer the Chebyshev basis, exactly as the D.3 results show.

Summary of changes in the revision

  1. Introduction (p. 1). Reworded the PINN sentence from "do not transfer" to a per-instance-vs-amortized distinction. How: the clause "and do not transfer to new inputs" is replaced by "transfer to new inputs only through additional training or a dedicated parametric construction, for example by meta-learning the weight initialization [1] or by one-shot transfer learning [2]", followed by "a neural operator, by contrast, amortizes the map f↦uf\mapsto u once and evaluates new inputs in a single forward pass". Two references added to references.bib.
  2. Section 3.2 (p. 4). Stated that u^i\hat{u}_i is the reference finite-difference solution. How: in the "Supervised stage" paragraph the definition of the training pairs now reads that each u^i(y)\hat{u}_i(y) is "the reference numerical solution of the PDE with forcing h^i\hat{h}_i, obtained by the high-fidelity solver of Appendix B.1 (a second-order five-point finite-difference discretization with Dirichlet conditions on a 101×101101\times101 grid for the 2-D benchmarks, solved by a direct sparse solve)".
  3. Section 4.1 (p. 6). Defined the spectral exponent at first occurrence and renamed it αtrain/αtest→βtrain/βtest\alpha_{\mathrm{train}}/\alpha_{\mathrm{test}} \to \beta_{\mathrm{train}}/\beta_{\mathrm{test}} throughout to remove the clash with the expansion coefficients αi\alpha_i. How: a parenthetical was inserted after the first use of βtrain/βtest\beta_{\mathrm{train}}/\beta_{\mathrm{test}} giving the spectral-density definition E[∣f^k∣2]=(1+∣k∣2)−β\mathbb{E}[|\hat{f}_k|^2]=(1+|k|^2)^{-\beta} with a pointer to Appendix B.2; every occurrence of the smoothness symbol in Sections 3–4, the table captions, and Appendix B.2 was changed to β\beta.
  4. Table 1 (p. 7) and the corresponding linear-NS rows of Tables 3–6, 14–16. Relabeled the linear NS rows "Stokes"; Section 4.1 states the linear benchmark is steady Stokes. Appendix B.1 (p. 16) now labels the linear form explicitly. How: the \multirow label "NS." was changed to "Stokes" in the linear-equation tables, and a sentence was added next to the Stokes/NS equation stating which system defines the linear benchmark.
  5. Section 4.5 (p. 8). Added the basis–input-distribution alignment comment. How: inserted a sentence explaining that the Fourier-basis advantage is an alignment between the basis and the periodic, band-limited GRF inputs (truncation error TNT_N decays spectrally), and that mismatched inputs (non-periodic Matérn fields, Appendix D.3) favor the Chebyshev basis, tying this to Theorem 3.1.

References

[1] Penwarden, Michael, Shandian Zhe, Akil Narayan, and Robert M. Kirby. "A Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEs." Journal of Computational Physics, Vol. 477, 111912, 2023. arXiv:2110.13361.

[2] Desai, Shaan, Marios Mattheakis, Hayden Joy, Pavlos Protopapas, and Stephen Roberts. "One-Shot Transfer Learning of Physics-Informed Neural Networks." arXiv preprint arXiv:2110.11286, 2021.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.