- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces the Twisted Multiparameter Neural Operator (TMNO), a spectral neural operator motivated by twisted multiparameter singular integral theory. The main idea is to move beyond Cartesian frequency representations by incorporating coupled frequency directions, particularly the diagonal component , alongside modular lattice shearing operations.
The architecture combines a tri-axial spectral filter, conical phase modulation, and spatial tube convolutions using shears. The authors evaluate the approach on thermal convection, Helmholtz scattering, compressible shock reflection, and Burgers equations. They report improvements over several neural operator baselines, including better generalization to rotated shocks and substantially fewer parameters than a FLOP-matched FNO.
The broader motivation is interesting: incorporating geometric and harmonic-analysis structure into neural operators to better represent directional singularities and multiscale PDE dynamics.
Critical strengths
I really like the general direction of bringing more sophisticated harmonic analysis into neural operator design :) Most spectral architectures are built around relatively simple Fourier parameterizations, so exploring coupled frequency structures and directional singularities is interesting, especially for PDEs with sharp interfaces and anisotropic dynamics.
The modular shearing idea is also appealing. Using integer lattice transformations to realign directional features without interpolation is mathematically clean, at least on compatible periodic grids. I appreciate that the authors evaluate multiple physical systems and include component ablations, rotated-shock experiments, and resolution-transfer results rather than relying on a single benchmark.
Critical weaknesses
1. I have significant concerns about the mathematical and physical consistency of the benchmarks.
One thing that immediately stood out to me is the Boussinesq setup in Appendix C.1. The authors define the operator input as , while also imposing the boundary condition along the bottom wall. If this notation refers to the spatial boundary, the input appears to be constant across samples. If instead it refers to the initial temperature field, the notation needs to distinguish spatial coordinates from time. Either way, the precise mapping being learned is unclear.
I also noticed that the Burgers initial conditions in Appendix C.4 are functions of . For , the constant-phase fronts lie along , whereas the paper repeatedly describes them as aligned with . These are different directions! Since the proposed architecture is motivated specifically by geometric alignment with singularities, I think this deserves careful clarification.
The compressible Euler setup raises another question: the stated input is only the initial density field, but the evolution generally also depends on the initial velocity and pressure. Unless these quantities are fixed or uniquely determined by the dataset construction, density alone does not specify the evolution.
2. The theoretical claims seem stronger than what the architecture and experiments establish.
The paper repeatedly describes its representation as resolving singularities through a rigorous connection to twisted multiparameter harmonic analysis. However, I am not convinced that the continuous singular integral estimates establish corresponding approximation or discretization guarantees for the learned neural operator.
For example, the tri-axial Calderón construction motivates the spectral parameterization, but the implemented model uses learned finite-band profiles and a discrete sector modulator. What guarantees survive this transition? In particular, I would distinguish the mathematical validity of the underlying multiplier construction from any claim that the trained network necessarily preserves physical discontinuities or avoids Gibbs-type errors.
I also found the 936× parameter-efficiency claim difficult to interpret. The comparison increases the FNO channel width substantially to obtain a particular FLOP count, producing a model with billions of parameters. This does not necessarily demonstrate a 936× practical efficiency advantage over a well-tuned FNO at comparable accuracy or computational budget. I would prefer a parameter/accuracy and runtime/accuracy comparison across realistically trainable models.
Overall, I think the experiments are potentially interesting, but the ambiguities in the PDE setups and the gap between the harmonic-analysis theory and learned-model guarantees substantially weaken my confidence in the conclusions.
Questions for authors
-
PDE setup: In Appendix C.1, what exactly is the input to the Boussinesq operator? The paper writes , but also fixes at the bottom boundary. How are different input samples generated?
-
Shock orientation: For the Burgers initialization in Eq. C5, gives constant-phase fronts along , not . Could the authors clarify this orientation convention and how it relates to the chosen shears and singular variety?
-
Theoretical guarantees: Which properties of the continuous twisted multiparameter operators are actually preserved by the learned discrete architecture? I would be interested in whether there are any approximation or stability guarantees beyond the underlying harmonic-analysis identities :)
-
Efficiency comparison: Why is the 936× parameter advantage established by expanding FNO to 1.87B parameters? Would a comparison against parameter-matched and FLOP-matched FNO variants at comparable accuracy provide a more meaningful measure of efficiency?
-
Conservation and boundaries: Proposition 3.1 establishes energy preservation for the lattice permutation itself. Does this extend in any form to the full TMNO layer? Also, how do the guarantees change for the padded, non-periodic domains used in the physical benchmarks?
6 comments
Response part 2
W2 & Q3–Q5. Theoretical scope, clarification/recall of the 936x figure, and conservation/boundary properties
-
Which continuous properties survive in the discrete learned architecture (W2 & Q3, Appendix B.6): We added a dedicated section, Appendix B.6 ("Properties Inherited by the Discrete Layer", page 17, lines 879–912), drawing a strict boundary between mathematical guarantees of the discrete layer and empirical observations:
- Guaranteed for all parameter values on the grid:
-
Real-valued outputs: The Hermitian projection on the self-conjugate columns ( and ) ensures real inputs produce exact real outputs.
-
Exact lattice automorphism: Each sheared branch is an exact Fourier multiplier on the original periodic grid without spatial interpolation error (Proposition 3.1).
-
Hard-limit reproduction of : As temperatures with , and , the discrete parameterization reproduces at every grid wavenumber off the Nyquist planes (whereas at training temperatures , the learned symbol is a smoothed member of this family).
- Explicit per-layer operator norm bound (Eq. B5): Because the band and sign memberships form convex partitions of unity, each linear branch satisfies:
What is explicitly NOT guaranteed (lines 908–912):We explicitly state that we do not claim universal approximation rates for the trained network, nor that a trained symbol remains close to or the continuous Calderón pieces, nor that the discrete band-limited network is mathematically guaranteed to preserve discontinuities or avoid Gibbs oscillations by construction; any reduction in oscillations observed in Figure 3 is empirical.
-
- Guaranteed for all parameter values on the grid:
-
Clarification and replacement of the
936x/ 1.87B-parameter FNO comparison (W2 & Q4, Sections 3.2, 4.5 & Appendix B.5):- How the original
936xand17.64 GFLOPsnumbers were obtained (and their exact caveats): Those figures are an exact, reproducible consequence of equating spectral FLOPs between TMNO and FNO under PyTorch'sFlopCounterMode(reported as Convention A in Appendix B.5, which counts a complex multiply–accumulate as a single real one, i.e., 2 FLOPs, and counts FFTs as zero). Under Convention A, TMNO ( on a grid; parameters) measures exactly ( in spectral mixing across all modes). Because a truncated FNO () applies its weights only on the corner blocks ( real spectral parameters), raising FNO's channel width to to match TMNO's Convention-A FLOPs yields and parameters—which is the nominal parameter budget (and TMNO's exact parameter count). - Why we agreed with your critique and replaced this ratio in the revised manuscript: We fully agree with the reviewer that inflating an FNO to ( parameters) just to match TMNO's all-mode FLOP count does not reflect a realistic efficiency advantage over a well-tuned, trainable FNO. In the revised manuscript, we have recalled the
936xheadline claim and replaced it with a direct Accuracy Versus Cost analysis across realistically trainable models (Section 3.2, Section 4.5, and Appendix B.5, Table B1), reporting both Convention A (FlopCounterMode) and Convention B (full real arithmetic: 8 FLOPs per complex MAC and per FFT) alongside wall-clock runtimes on an NVIDIA RTX A6000 GPU ( grid, batch size 4):
- How the original
| Model Architecture | Parameters | GFLOPs (Conv. A) | GFLOPs (Conv. B: Full + FFT) | Step Time [s] | 150 Epochs [min] |
|---|---|---|---|---|---|
| TMNO (Full Model) | |||||
| TMNO, identity frame only () | |||||
| FNO (, pm) | |||||
| FNO (, pm) |
Response part 3
As we now state explicitly in the Abstract, Section 3.2, and Section 4.5, TMNO's parameter count () is independent of the number of retained Fourier modes, whereas its computational cost ( under Convention A / under Convention B) is substantially higher than that of parameter-matched FNOs ().
- Energy conservation of the full layer vs. gathers & non-periodic boundary handling (Q5, Section 3.3 & Appendix B.6):
- Does Proposition 3.1 extend to the full TMNO layer? No. As clarified in Section 3.3 (lines 266–269) and Appendix B.6 (lines 893–902), Proposition 3.1 ( energy conservation / unitarity) applies strictly to the lattice gathers alone (proving they permute grid nodes without numerical dissipation or interpolation error). The full TMNO block is not norm-preserving: the learned band profiles, sector table, channel mixing matrices , pointwise skip connection, group normalization, and GELU activation all change the norm. Energy conservation holds for only if its symbol is unitary at every mode, which is not enforced; instead, the linear spectral branch satisfies the bounded operator norm estimate in Eq. (B5).
- How do guarantees change on non-periodic domains? As stated in Section 3.3 (lines 281–285) and Appendix B.6 ("Non-periodic data", lines 904–907), the lattice gathers are exact automorphisms only on square periodic grids () and are implemented via periodic modular wrap-around without spatial padding. Consequently, on the non-periodic benchmarks (Compressible Euler in Appendix C.2 and Helmholtz scattering in Appendix C.3), every branch couples opposite boundaries of the computational domain—a modeling approximation shared with the standard unpadded spectral baselines (FNO, TFNO, U-NO) evaluated under the same protocol.
Response part 1
We are deeply grateful to reviewer for the encouraging words on our core research direction—especially your appreciation for bringing multiparameter harmonic analysis, coupled frequency structures, and interpolation-free SL2(Z) modular lattice shearing into neural operator design, as well as our multi-system ablations and rotated-shock evaluations. Your careful audit of our PDE benchmark descriptions, orientation conventions, and theoretical claims pinpointed exactly where our initial exposition fell short of the mathematical standard we aimed for. We have thoroughly revised the manuscript so that every PDE setup is stated with complete physical precision and every theoretical claim aligns strictly with what the discrete architecture guarantees.
W1 & Q1–Q2. Mathematical and physical consistency of the PDE benchmarks
Boussinesq / Buoyancy-driven Navier–Stokes setup (W1 & Q1, Appendix C.1): The ambiguous notation a(x)=T(x,0) and T(x,0)=1 in the earlier draft has been completely replaced with the exact specification of the PDEArena conditioned 2D buoyancy-driven Navier–Stokes dataset (Gupta & Brandstetter, 2022) in Appendix C.1 (lines 923–931):
Burgers shock orientation convention (x1+x2=const vs. x1=x2) and action of SL2(Z) shears (W1 & Q2, Section 3.1 & Appendices C.4, C.6): The reviewer caught a crucial geometric distinction, which we now state explicitly in Section 3.1 ("What the frames do", lines 229–237) and Appendix C.4 (lines 966–969):
TMNO-U, both diagonalsin Table 2 and Table D9 (which includes SL2(Z) shears for both diagonal families i−j=const and i+j=const, achieving 0.0043 on Burgers ID, 0.1697 on held-out angles, and 0.3977 on further angles), and added the Controlled Burgers Orientation Study (Appendix C.6, Table C1) evaluating 12 held-out primitive lattice directions across both diagonal families.Compressible Euler state variables (W1, Appendix C.2): We updated Appendix C.2 (lines 933–946) to specify the exact
euler_multi_quadrants_openBCdataset from The Well (Ohana et al., 2024) solving the 2D compressible Euler equations with adiabatic index γ=1.13 (40 training/validation trajectories and 10 test trajectories). The operator input is not density alone, but the full 5-channel physical state—density ρ, total energy E, pressure p, and x- and y-momentum (ρu,ρv)—at the first stored snapshot (box-averaged from 5122 to 1282), predicting the full 5-channel state at the next snapshot (Δt=0.015).