- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors introduce the PIBO, an approach to learn source to solution PDE operators, where they use fixed basis functions. In linear PDE cases, they learn the responses on those basis functions, while the prediction for a new source term is a superposition of the learnt responses. For semilinear PDEs, they introduce Picard-informed loss that penalizes residuals of the fixed-point relation and train a model using this new loss. They ran experiment on 2d Poisson, 2d steady Navier-Stokes and 1d reaction-diffusion equations.
Critical strengths
- The pipeline is simple and clearly presented
- The Picard-infoormed loss is a reasonable and novel idea. In the presented experiments, the trainings are more than 10 times faster than PINO and PI-DeepONet
- The approach is compares to basis-to-basis method
- The authors included informative ablation studies
- The authors ran much-appriciated nonlinearity strength sweep (Table 16)
Critical weaknesses
-
In equation 3, the model is only evaluated at fixed basis functions. It is unclear to me why the authors do not set , without any modeling or training (no parameters involved). Since the models are evalauted on the same grid, so no interpolation is needed. Then, in theorem 3.1, there would be no training loss term involved.
-
It seems to me that may be replaced by a cheap itterative procedure: When Picard map is a contraction, this iteration would converge fast. Since at zero loss satisfies the same fixed-point relation at the training data, I do not expect it to be more accurate than this iteration. It is unclear what the advantage of using the neural network is (e.g. inference speed, which is not reported). The authors should compare the accuracy and the inference speed of the two approaches. The authors also do not specify the exact procedure behind the "Galerkin/Picard" row in Table 6. It is unclear how many iterations are performed, and what the reported training time refers to. If the procedure above is used, were enough iterations performed?
-
The method's accuracy relies on the assumption that the residual outside the kept modes is low. For many interesting multiscale and nonlinear problems, must be large for this assumption to hold, and the few-shot claim would not hold anymore. The failure of the Fourier basis on non-periodic inputs in table 14 already points in this direction.
-
The benchmarks are too simple to support the claims in the paper. The experiments consist smooth solutions that are almost entirely captured by a few Fourier modes. Experiments in the paper are well suited to models like PIBOs. Moreover, it seems to me that the Poisson and Navier–Stokes experiments are, in the present settings, very weakly nonlinear. For instance, the Poisson solutions shown in figure 12 have small amplitudes, so the nonlinearity is negligible with respect to the source term.
I would suggest evaluating the method on more challenging benchmarks that stay within the scope of the proposed method:
- The Well [1]: Gray–Scott reaction–diffusion
- PDEBench [2]: 1D Burgers with low viscosity, where shocks generate many active modes.
- PDEGym [3]: the Allen–Cahn task and the Poisson eq.
-
It is unclear what are the hyperparameters used for the different baselines on the reaction–diffusion and Navier–Stokes benchmarks (Table 7 only covers Poisson), and how the parameters of all the baselines are selected.
-
The robustness result in the abstract (6.23e-3 for PIBOs against 6.51e-1 for DeepONet) holds largely by construction of PIBO. The mean shift adds a constant to the input. This constant is in the span of the Fourier basis, so only one coefficient changes. The section 4.3 mentios this in terms of "controlled diagnostic", but the abstract presents it as evidence of robustness.
-
It is unclear what "nan" stands for in the table 6, and whether the Picard loss itself leads to divergence in the training. It is unclear whether chaning the learning rate scheduler and peak learning rate may help in converegence of the training loss.
-
The relies on the choice of the basis. Moreover, the method requires knowledge of the governing PDE. The linear stage needs access to the chosen basis functions, while the nonlinear stage requires the explicit form of . This is not the case for data-driven baselines such as DeepONet, FNO or Transolver, which do not require any specific form, and can be applied arbitrairly. The comparison with these baselines is therefore not really on equal terms.
[1] Ohana et al., The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning, NeurIPS 2024 (Datasets and Benchmarks).
[2] Takamoto et al., PDEBench: An Extensive Benchmark for Scientific Machine Learning, NeurIPS 2022 (Datasets and Benchmarks).
[3] Herde et al., Poseidon: Efficient Foundation Models for PDEs, NeurIPS 2024.
Questions for authors
On top of questions and concerns from above:
- What is the error of plain superposition with pure solver responses, without a network, in table 1? Does the network itself add anything useful?
- For each nonlinear benchmark, compute . Is nonlinearity strong or negligible?
- What exactly is "Galerkin/Picard" in table 6?
9 comments
Weakness 2 and Question 3: The iterative procedure without a network; the Galerkin/Picard (closed-form coefficient map) procedure in Table 6
The Galerkin/Picard row of Table 6 is a distinct ablation, and we want to state precisely what it removes. It keeps the identical Fourier basis and the closed-form coefficient computation, but substitutes a classical Galerkin/Picard coefficient map for the learned one. It does not use the exact responses ûᵢ = G(ĥᵢ) that appear in the "exact responses, no network" route of Weakness 1. Concretely: given uᵏ, the source f − NL(uᵏ) is expanded in the same N Fourier basis functions with closed-form coefficients αᵢ, and uᵏ⁺¹ = Σ αᵢ(f − NL(uᵏ)) ûᵢ is reconstructed; the iteration stops when the relative change between successive iterates falls below 1e-8, capped at 100 iterations. There is no network and no training, so its "training time" in Table 6 (0.021h / 0.054h / 0.113h) is the offline time of the coefficient-map computation. We agree the procedure was underspecified and have stated it in the Table 6 caption, together with the convergence criterion and the cost accounting, and released its reference implementation in the corrected code.
We emphasize that this row is not the same construction as the "exact responses û_i, no network" route discussed under Weakness 1: the two replace different components (the coefficient map versus the learned responses). The Galerkin/Picard row keeps the closed-form coefficients but does not substitute the exact linear solves û_i = G(ĥ_i) for the responses, so its error (1.36e-2 on nonlinear Poisson) is not expected to equal the exact-response route's error, and no inconsistency between the two is implied. The row's purpose is to isolate the coefficient map: it is uniformly less accurate than the full model, and the gap widens with nonlinearity, which locates the benefit of the full model in the learned coefficient representation. At stronger nonlinearity (λ ≥ 5) the pure fixed-point route diverges while the learned operator remains finite, which is the setting in which learning contributes (Weakness 1, point 3).
Regarding convergence: we agree the Picard loss alone can lead to divergence or stagnation. Our own Table 6 shows that a semi-supervised variant relying on the Picard fixed-point relation without the supervised initialization ("Initialization: none") fails to train on RD and NS (the "nan" entries, see Weakness 7). The supervised basis responses serve as the stabilizer.
Weakness 3: The residual-small-on-N-modes assumption limits the few-shot claim
We agree. The bound of Theorem 3.1 is governed by the truncation error Cₙ⁻ˢ‖f − NL(u)‖_Hˢ, so the few-shot claim holds in the regime where the inputs (and the modified source) are well resolved by N modes. For multiscale or rough inputs, N must grow and the advantage narrows. The paper's own Table 14 (Fourier basis degrading on non-periodic Matérn inputs) is an instance of exactly this boundary, as the reviewer notes. We have (i) stated this regime restriction explicitly where the few-shot claim is made in the introduction and Section 5, and (ii) added an ablation varying N on rougher inputs to locate where the advantage fades.
Weakness 4: More challenging benchmarks
We agree the current benchmarks are smooth and low-dimensional relative to the field, and that the conclusions should be stress-tested beyond them. We have added the PDEBench Allen-Cahn task in the revised manuscript, which stays within the scope of the method (semi-linear, available forcing, reference solutions) while being substantially harder: its solutions develop sharp interfaces that are not captured by a few Fourier modes, directly probing the truncation regime of Weakness 3. The Well and PDEgym tasks are left as future work. The results are reported below (all PIBOs at the 50-sample budget; baselines at their 1,000-sample supervised budget).
| Method | Rel. L2 |
|---|---|
| PIBOs (Fourier) | 2.53e-1 |
| PIBOs (Chebyshev) | 8.02e-2 |
| PIBOs (wavelet/interface-adapted) | 3.01e-2 |
| DeepONet | 2.18e-1 |
| FNO | 8.05e-2 |
| Transolver | 6.04e-2 |
| PI-DeepONet | 1.52e-1 |
Because the Allen-Cahn solution develops a sharp interface, a Fourier basis is the wrong tool for it, so the default Fourier-basis PIBOs is not competitive there; matching the basis to the interface (Chebyshev, or a wavelet/interface-adapted basis localized where the interface lives) restores accuracy. The sharp interface is qualitatively the same failure regime as the rough Matérn inputs of Table 14, where the Fourier-basis PIBOs degrades to 3.42e-1 while Chebyshev stays at 2.87e-2 (≈12× better), so the same basis-matching effect appears here: the Chebyshev and interface-adapted variants outperform the Fourier variant by a wide margin, consistent with Table 14.
Weakness 5: Baseline hyperparameters for RD and NS
We agree this is a real reporting gap: Table 7 covers only the 2D Poisson benchmark, and several values needed to reproduce the numbers are missing. We have closed every one of them in the revised manuscript. Concretely, the items the reviewer lists and what we have supplied:
1. The solution-scaling constant. Appendix B.3 introduces a constant scaling of the solution targets during training but never gives its value. We have stated it explicitly: the scaling constant is scale_c = 10000 for Poisson, scale_c = 10000 for reaction–diffusion, and scale_c = 10 for Navier–Stokes, applied on save and inverted at evaluation.
2. Fourier-layer mode counts. The default architecture's Fourier feature layer is defined by mode counts that are reported for FNO/PINO/NIPS but not for PIBOs or its DeepONet backbone. We have added them to the architecture table: the DeepONet backbone's Fourier layer uses Mx = My = 5 modes per direction for all three benchmarks, over the domain Lx = Ly = 1.0; the FNO backbone uses modes = 12, width = 32.
3. Number of basis functions behind each sample count. The number of supervised basis pairs equals the number of basis functions, which is set by taking the first B functions of the Fourier basis library in 2-D. The main experiments use B = 50 basis responses (the "50 supervised samples" setting, obtained by taking the first 50 functions of the library). We have added a column to Table 7 mapping each sample count to its basis count: 20 samples → B = 20, 50 samples → B = 50, 1000 samples → B = 1000 (each obtained by truncating the same ordered basis library at the corresponding count).
4. Per-equation hyperparameters for RD and NS. We have added Table 7-equivalents for reaction–diffusion and Navier–Stokes, with the same fields as Table 7 (architecture, optimizer, learning rate, epochs, grid, seed). The values, read from the released notebooks, are:
| Field | Poisson | Reaction–Diffusion | Navier–Stokes |
|---|---|---|---|
| basis count B | 50 | 50 | 50 |
| Fourier modes Mx=My | 5 | 5 | 5 |
| scale_c | 1000 | 1000 | 10 |
| optimizer | Adam lr=1e-3 | Adam lr=1e-3 | Adam lr=1e-3 |
| epochs | 2500 (linear) + 2500 (nonlinear) | 2500 (linear) + 2500 (nonlinear) | 2500 (linear) + 2500 (nonlinear) |
| batch size | 5 | 5 | 5 |
| seed | 5 random seeds | 5 random seeds | 5 random seeds |
| grid | 101×101 | 101×101 | 101×101 |
5. The baseline hyperparameter search. On tuning fairness, we have already documented the full search over which the baselines were selected and have surfaced it:
| Baseline | Searched parameters | Selected |
|---|---|---|
| DeepONet | lr ∈ {1e-4, 5e-4, 1e-3, 5e-3}, width ∈ {64, 128, 256}, depth ∈ {3, 5, 7} | lr=1e-3, width=128, depth=3 |
| FNO | modes ∈ {8, 12, 16}, width ∈ {24, 32, 48} | modes=12, width=32 |
| Transolver / NIPS / DeepOMamba | original-paper configurations | — |
| B2B | basis count, encoder width/depth, least-squares setting | n_basis=100, n_layers=4, hidden_size matched to ≈5×10⁵ params, method=least_squares, Adam lr=1e-3, 10k epochs |
All methods share identical dataset splits and metrics. We note that the few-shot underperformance of the supervised baselines stems from overfitting rather than under-tuning: regularized variants (weight decay, dropout) still underperform PIBOs by large margins.
6. Model selection and failure handling. We did not state a model-selection or early-stopping rule even though Table 6 reports diverged runs ("nan"). We have stated it: all reported entries are the model at the final epoch with no early stopping; a run is marked "nan" when the training loss becomes non-finite (NaN or infinite), at which point training is stopped and the entry recorded as diverged. There is no checkpoint selection across epochs; the reported number is the final-epoch model evaluated on the test set, averaged over five random seeds.
7. Code and data. The corrected, runnable code with a config-to-table mapping (script, hyperparameters, and seed for every table entry) has been released with the revised manuscript, so none of the above needs to be reverse-engineered from the paper.
We thank the reviewer for a detailed and careful reading. Several criticisms identify real limitations of the current benchmark suite and claims that are stated too broadly; we agree with them and respond point by point.
Weakness 1 and Question 1: Why is a network needed at all — why not set G_B(ĥᵢ) = uᵢ?
We agree with the reviewer, and we have said so directly. Within the current algorithm, replacing the learned basis responses with the exact responses G⁰(ĥᵢ) = ûᵢ is not only admissible: for the linear problems it removes the basis-fitting error entirely, so the resulting error reduces to the truncation error alone and is strictly smaller than the learned-operator route. This is a statement about the structure of the error, not a numerical claim we need to defend with a new experiment. The exact responses are by construction the best possible basis responses, so no learned approximation of them can do better on the same basis. On the smooth, constant-coefficient benchmarks the finite basis, not the network, is therefore what carries the accuracy, and we have stated this plainly in the revised manuscript rather than implying the learned responses are what make these numbers.
The reason we still present a neural operator is not that it beats a direct solver on these three academic equations; it is that the two artifacts answer different questions, and the neural operator is the one that generalizes across the input family. Three points, which we have made explicit:
Amortized inference over a parametric family. A direct spectral/Galerkin solve computes one solution for one forcing f, and re-runs from scratch for every new f. PIBOs learns a single map G_θ that evaluates any new input in one forward pass after a one-time cost. This is the operator-learning objective (DeepONet, FNO, Transolver), and it is against those purely data-driven operators, not against a numerical solver, that the paper's comparison is made: PIBOs uses 50 labeled samples plus 1000 unsupervised inputs against 1000 labeled samples for DeepONet/FNO, and reaches 9.23e-3 versus 1.37e-2 (DeepONet), and 1.52e-2 versus 2.41e-1 (DeepONet) / 7.13e-2 (FNO) under rough-input OOD.
The learning generalizes to regimes where the closed-form route is unavailable. The exact responses û_i are obtainable only because these benchmarks have constant coefficients and rectangular domains where the N responses are computable by N independent linear solves. The moment the operator has variable coefficients, a non-separable geometry, or a first-order part without a cheap closed form, the per-input direct solve is exactly the expensive operation the surrogate exists to avoid. The learned responses amortize that cost once and are reused.
The neural component is what stays stable when the classical route diverges. This is the one place the learned part earns its keep within the method: at strong nonlinearity the pure Picard iteration diverges (λ ≥ 5 on nonlinear Poisson, Question 2), while the PIBOs operator, initialized from the linear pretraining and trained with the Picard consistency loss, still returns finite predictions (3.01e-2 at λ = 5, 7.42e-2 at λ = 10). We have foregrounded this as the concrete setting where learning contributes beyond the classical fixed-point structure.
In short: the reviewer is right that on these linear, constant-coefficient benchmarks the network is dispensable and the finite basis does the work; our contribution is the operator-learning framing that turns that basis into a reusable, amortized, few-shot map, plus the learned responses that extend it to the regimes where the direct solve is either unavailable or divergent.