- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a novel spectral neural operator—BALANI-NO (Basis-Adaptive Learned Asymptotic Nonlocal Inversion Neural Operator)—designed to tackle operator learning problems for elliptic partial differential equations (PDEs) surrogates and strong gravitational lensing.
Core Architectural Innovation: Traditional spectral operators (such as FNO) typically parameterize the Fourier multiplier (symbol ) in each layer using a learned lookup table over the lowest modes. However, for many elliptic operators, the symbol is instead a rational function of frequency. BALANI-NO innovatively parameterizes the symbol as the sum of two components: 1.A learned pole (rational) branch in the isotropic magnitude (specifying the symbol at all frequencies from a few coefficients, favoring no coordinate axis and remaining independent of the grid); 2.A small low-mode traditional Fourier table (providing a complementary low-frequency correction). Experiments and Evaluation: Under matched parameter budgets and across 7 different random seeds, the authors extensively compared BALANI-NO against 10 mainstream baseline models (including FNO, ALNO, LNO, PDNO, various U-shaped operators, CNNs, and DeepONets).
Critical strengths
1.Strong Physics-Informed Inductive Bias: Tailored for elliptic PDEs and inverse problems with physical backgrounds (such as the lensing operator corresponding to the rational form ), the paper bypasses pure black-box lookup tables by incorporating an isotropic rational pole branch. This design naturally aligns with the asymptotic behavior of many physical operators. 2.Exceptional Zero-Shot Cross-Resolution and Cross-Distribution Generalization: The standout highlight of this paper is its remarkable zero-shot transfer performance. Whether dealing with resolution changes (Darcy, Navier-Stokes) or shifts in physical dark matter models (WDM CDM/Axion), BALANI-NO exhibits extremely low degradation, outperforming most flexible baselines that tend to break down under grid variations.
Critical weaknesses
1.Training-Resolution Performance Is Not Always Optimal: As the authors honestly note in the discussion and Table 7, while BALANI-NO demonstrates dominant performance in zero-shot settings, it is not always the winner at the training resolution (e.g., losing to CNNs and PDNO on Darcy flow). This occurs because its constrained global symbol parameterization sacrifices some single-grid fitting flexibility.
2.Limitations on Non-Input-Invariant / Variable-Coefficient Operators: When the underlying physical operator varies spatially with the input (such as the spatially varying diffusion coefficient in Darcy flow), a single global symbol does not exist. Consequently, BALANI-NO's advantages in such tasks lie primarily in generalization rather than training-set fitting.
3.Computational and Implementation Complexity: Although parameter budgets are matched, it remains unclear whether introducing softplus constraints and polynomial combinations for the rational pole branch significantly increases training time or introduces numerical instability (such as higher-order poles collapsing or weak identifiability, as mentioned in the appendix regarding the second pole's larger uncertainty). The paper lacks a direct wall-clock training time comparison.
Questions for authors
1.Clarification Request: In the rational pole branch defined in Eq. (4), pole locations are determined by , with a separate learned offset introduced at . During training, how sensitive is the choice of initial pole locations (linearly spaced in in the code) to converging to specific physical screening lengths when dealing with different physical tasks (such as the diffusion term in Navier-Stokes versus the rational operator in gravitational lensing)? Do poles ever get stuck in local minima or degenerate (e.g., multiple poles collapsing to the same value)?
2.Clarification Request: Regarding computational efficiency, while BALANI-NO maintains a comparable parameter count to the baselines, how do its memory footprint and computational throughput (during training and inference) compare to FNO or LNO, given that it computes full-frequency rational functions rather than FNO's truncated window?
3.Suggestion for Future Work: The authors observed in Darcy flow and complex real-world data (like EMAG2) that locally grid-tied models (such as CNN/PDNO) fit better at the training resolution but degrade during cross-resolution transfer. Could future work explore an "adaptive hybrid weight mechanism" that permits more flexible local corrections near low resolutions/training grids, while smoothly transitioning to BALANI-NO's global rational pole-dominant mode in high-resolution/zero-shot regions?
4 comments
Response to Reviewer wTJe (2/2): pole initialization and dynamics, future work
Q1. Sensitivity to initial pole locations; stuck or collapsing poles.
We thank the reviewer for this question. We now study it directly in the new Appendix B.2, on a screened-Poisson task with a known true pole () and on Allen–Cahn (), which has none.
- Default training. The poles stay within 9% of their initial values and remain well separated (minimum gap 0.17). The softplus is the cause: at the smallest initial value, , its derivative is , so these poles receive gradients about 400 times weaker than an unconstrained parameter. The optimizer therefore reduces the loss mainly through the residues, and the pole branch acts as a fixed multi-scale rational basis with learned residues. The main-task models behave the same way: the block-0 poles of the Navier–Stokes and full-field lensing models move by at most 0.01 and 0.024 in absolute terms.
- Convergence to a physical scale. For a single linear pole layer the scale is identifiable: fitting by grid search recovers known screening lengths within 0.11% for one pole, and within 0.42% for the smaller of two poles (Appendix B.1, Table 10). In the full three-block network it is not. Even with a separate pole learning rate of , no block-0 pole reaches the true value 4 (the largest reaches 2.62), because lifting, projection and pointwise nonlinearities between blocks mean that no single block needs to hold the true pole.
- Collapse. Under the default optimizer we see no collapse. With a pole learning rate, two poles in block 1 of the Allen–Cahn model do collapse (2.7384 and 2.7372), so freely moving poles can degenerate.
- Initialization. The initial range matters. On screened Poisson the error ranges from 0.0049 to 0.0098 across five initializations (Table 12). Starting from larger poles, closer to the true scale, reduces the error by 36% relative to the default, and a separate pole learning rate reduces it by up to 21% (Table 11).
On the second pole's larger uncertainty that the reviewer mentions: in Table 10 it comes from overlapping contributions. Once the first pole captures the dominant low-frequency behavior, the second is less well determined, while the test error stays below .
Following this analysis, we have revised contribution (i) and the Limitations to describe the pole branch as a multi-scale rational basis with learned residues and trainable pole locations. To keep the comparison fair, the main tables use the same default initialization and optimizer on every task, and none of the improved settings above.
Q3. Adaptive hybrid weighting between local corrections and the global pole branch.
We thank the reviewer for this suggestion and have added it to the Conclusion as future work: a learned, resolution-aware weighting between the global pole branch and local corrections. The local part would need to be defined independently of the grid, since the grid-tied flexibility of CNN and PDNO is exactly what makes them degrade under a resolution change.
We hope these additions address the reviewer's concerns and would be glad to clarify any remaining points.
The revision is responsive, the method is simple, cheap and reproducible, and the empirical protocol — eleven architectures at matched parameter budgets, seven seeds, six tasks including two on real data — is more thorough than is typical at this venue. The new appendices add scientific content rather than filler, and the honest reporting of negative results is a genuine service to the community. My remaining objections concern claim calibration, one missing control, and one self-imposed handicap in the protocol; none requires new theory or a new dataset, and all can be handled in a camera-ready.
Three changes would move me from weak accept to accept: (i) the M2 reframing in the abstract and Section 4.3, so that the reader learns from the abstract, not from a footnote, that the pole branch does not beat a matched table on full-field lensing; (ii) the M3 re-run under the better pole setting, or an explicit statement that the adaptivity of the basis is nominal; (iii) the m1 paired tests on the headline claims, with ties labelled as ties.
Response to Reviewer wTJe: all three requested changes made
We thank the reviewer for the detailed follow-up. We have made all three changes in the revised PDF. This round's changes are marked in blue, and changes from the first revision remain in red.
(i) Full-field lensing in the abstract and the ablation. The abstract now states that the pole branch is needed on Navier–Stokes and subhalo lensing, while on full-field lensing a parameter-matched table-only model performs equally well. The ablation (Section 4.4) reports this in the main text with the paired differences (, and on WDM, CDM and Axion). The full-field convergence is dominated by the smooth main lens, so most of the target energy lies in the low modes that a wider table covers.
(ii) Adaptivity of the basis. We chose the explicit statement. Section 3 and the Limitations (now Appendix A) state that under the protocol used for all main results the pole locations move by at most from their initialization, so the adaptivity of the basis is nominal: the pole branch acts as a fixed multi-scale rational basis with learned residues. A separate pole learning rate and a task-informed initialization reduce the error on the screened-Poisson task in Appendix D.2, but we do not use them in the main tables, so that all architectures share the same training protocol.
(iii) Paired tests on the headline claims. A new appendix (Appendix B, Table 8) reports paired intervals (-test over 7 seeds, ) between BALANI-NO and the strongest baselines, with ties and losses labelled, and Section 4.3 points to it. All zero-shot headline claims hold. The difference (baseline minus BALANI-NO) against the closest baseline is:
| Task | Split | Closest baseline | Difference |
|---|---|---|---|
| Darcy | (zero-shot) | FNO | (win) |
| Navier–Stokes | (zero-shot) | FNO | (win) |
| Full-field lensing | CDM (zero-shot) | FNO | (win) |
| Subhalo lensing | CDM (zero-shot) | PDNO | (win) |
| Darcy | HC-UNO | (tie) | |
| Darcy | PDNO | (loss) | |
| Darcy | CNN | (loss) |
The ties and losses are the in-distribution Darcy comparisons already discussed in Section 4.
We believe these changes address the three points listed in your comment, and we thank you again for the careful reading.
Response to Reviewer wTJe (1/2): training-resolution trade-off, cost and stability
We thank the reviewer for the positive assessment of the physics-informed design and of the zero-shot transfer results, and for the detailed questions. Please see our general response for a summary of all revisions; changes in the revised PDF are marked in red.
W1. Training-resolution performance is not always optimal.
We agree. BALANI-NO is the training-resolution winner on Navier–Stokes and subhalo lensing, ties U-FNO on full-field lensing, and is 3rd on Darcy, 2nd on CAMELS and 4th on EMAG2 (Table 7). On those three tasks, CNN or PDNO wins at the training grid but degrades by 7.3–41.6× after a resolution change, while BALANI-NO has the lowest zero-shot error on every task. Section 5 and the Limitations discuss this trade-off. The revised Discussion also gives the reason for each task: Darcy has no global symbol, the CAMELS input is subsampled and does not determine the target, and the EMAG2 target pipeline includes a taper that is not shift-invariant (Appendix C).
W2. Operators that vary with the input (no single global symbol).
We agree, and the Limitations now state this explicitly for Darcy. The new Appendix A tests it in a controlled setting. On Allen–Cahn we vary only the strength of the nonlinearity ε, from pure diffusion (which has an exact symbol) to a strongly nonlinear map (which has none). For ε=0 the pole branch alone matches the full model (+0.0014±0.0051). For ε≥1 the table gives a significant gain, and the 95% paired intervals exclude zero (Table 9). The pole branch is needed at every ε: removing it raises the error by 2.3–3.6× (Table 8).
W3 / Q2. Training time, memory and numerical stability.
The new Appendix L measures training and inference cost for all eleven architectures (Tesla T4, batch 16, Navier–Stokes configurations; 20 timed repetitions after 3 warm-up runs). Selected rows from Table 23: