acceptodds
Under review as a conference paper at ICLR 2027

Twisted Multiparameter Neural Operators for Anisotropic Wave Dynamics

Abstract

Continuous neural operators have emerged as a prominent paradigm for learning resolution-independent mappings between infinite-dimensional function spaces in partial differential equations (PDEs). However, conventional spectral operators rely on Cartesian tensor-product multipliers that are inherently smooth across diagonal frequency rays, fundamentally limiting their capacity to represent oblique shocks, tilted shear interfaces, and anisotropic wave-fronts without severe parameter explosion. In this work, we introduce the Twisted Multiparameter Neural Operator (TMNO), a mathematically principled operator learning framework grounded in the harmonic analysis of twisted multiparameter singular integrals. By pushing forward a multi-parameter product structure through an algebraic quotient projection, TMNO parameterizes non-tensor singular multipliers concentrated along the coupled singular variety , unlocking a native tri-axial Calder\'on multiscale decomposition on two-dimensional domains. To physically align with directional transport without grid-interpolation dissipation or frequency aliasing, TMNO incorporates an automorphic tube branch governed by exact modular lattice shears, while structurally decoupling representational parameter capacity from computational FLOP throughput. Across diverse fluid convection, high-contrast acoustic scattering, and compressible shock reflection benchmarks, TMNO demonstrates superior multiscale front resolution, robust out-of-distribution generalization under rotated shock profiles, and orders-of-magnitude higher parameter efficiency.

Primary area:
learning theory
Keywords:
Neural Operator, Hilbert transform

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nTYNOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces the Twisted Multiparameter Neural Operator (TMNO), a spectral neural operator motivated by twisted multiparameter singular integral theory. The main idea is to move beyond Cartesian frequency representations by incorporating coupled frequency directions, particularly the diagonal component ξ1+ξ2\xi_1+\xi_2, alongside modular lattice shearing operations.

The architecture combines a tri-axial spectral filter, conical phase modulation, and spatial tube convolutions using SL2(Z)SL_2(\mathbb Z) shears. The authors evaluate the approach on thermal convection, Helmholtz scattering, compressible shock reflection, and Burgers equations. They report improvements over several neural operator baselines, including better generalization to rotated shocks and substantially fewer parameters than a FLOP-matched FNO.

The broader motivation is interesting: incorporating geometric and harmonic-analysis structure into neural operators to better represent directional singularities and multiscale PDE dynamics.

Critical strengths

I really like the general direction of bringing more sophisticated harmonic analysis into neural operator design :) Most spectral architectures are built around relatively simple Fourier parameterizations, so exploring coupled frequency structures and directional singularities is interesting, especially for PDEs with sharp interfaces and anisotropic dynamics.

The modular shearing idea is also appealing. Using integer lattice transformations to realign directional features without interpolation is mathematically clean, at least on compatible periodic grids. I appreciate that the authors evaluate multiple physical systems and include component ablations, rotated-shock experiments, and resolution-transfer results rather than relying on a single benchmark.

Critical weaknesses

1. I have significant concerns about the mathematical and physical consistency of the benchmarks.

One thing that immediately stood out to me is the Boussinesq setup in Appendix C.1. The authors define the operator input as a(x)=T(x,0)a(x)=T(x,0), while also imposing the boundary condition T(x,0)=1T(x,0)=1 along the bottom wall. If this notation refers to the spatial boundary, the input appears to be constant across samples. If instead it refers to the initial temperature field, the notation needs to distinguish spatial coordinates from time. Either way, the precise mapping being learned is unclear.

I also noticed that the Burgers initial conditions in Appendix C.4 are functions of q1x1+q2x2q_1x_1+q_2x_2. For q=(1,1)q=(1,1), the constant-phase fronts lie along x1+x2=constantx_1+x_2=\text{constant}, whereas the paper repeatedly describes them as aligned with x1=x2x_1=x_2. These are different directions! Since the proposed architecture is motivated specifically by geometric alignment with singularities, I think this deserves careful clarification.

The compressible Euler setup raises another question: the stated input is only the initial density field, but the evolution generally also depends on the initial velocity and pressure. Unless these quantities are fixed or uniquely determined by the dataset construction, density alone does not specify the evolution.

2. The theoretical claims seem stronger than what the architecture and experiments establish.

The paper repeatedly describes its representation as resolving singularities through a rigorous connection to twisted multiparameter harmonic analysis. However, I am not convinced that the continuous singular integral estimates establish corresponding approximation or discretization guarantees for the learned neural operator.

For example, the tri-axial Calderón construction motivates the spectral parameterization, but the implemented model uses learned finite-band profiles and a discrete sector modulator. What guarantees survive this transition? In particular, I would distinguish the mathematical validity of the underlying multiplier construction from any claim that the trained network necessarily preserves physical discontinuities or avoids Gibbs-type errors.

I also found the 936× parameter-efficiency claim difficult to interpret. The comparison increases the FNO channel width substantially to obtain a particular FLOP count, producing a model with billions of parameters. This does not necessarily demonstrate a 936× practical efficiency advantage over a well-tuned FNO at comparable accuracy or computational budget. I would prefer a parameter/accuracy and runtime/accuracy comparison across realistically trainable models.

Overall, I think the experiments are potentially interesting, but the ambiguities in the PDE setups and the gap between the harmonic-analysis theory and learned-model guarantees substantially weaken my confidence in the conclusions.

Questions for authors
  1. PDE setup: In Appendix C.1, what exactly is the input to the Boussinesq operator? The paper writes a(x)=T(x,0)a(x)=T(x,0), but also fixes T(x,0)=1T(x,0)=1 at the bottom boundary. How are different input samples generated?

  2. Shock orientation: For the Burgers initialization in Eq. C5, q=(1,1)q=(1,1) gives constant-phase fronts along x1+x2=constantx_1+x_2=\mathrm{constant}, not x1=x2x_1=x_2. Could the authors clarify this orientation convention and how it relates to the chosen shears and singular variety?

  3. Theoretical guarantees: Which properties of the continuous twisted multiparameter operators are actually preserved by the learned discrete architecture? I would be interested in whether there are any approximation or stability guarantees beyond the underlying harmonic-analysis identities :)

  4. Efficiency comparison: Why is the 936× parameter advantage established by expanding FNO to 1.87B parameters? Would a comparison against parameter-matched and FLOP-matched FNO variants at comparable accuracy provide a more meaningful measure of efficiency?

  5. Conservation and boundaries: Proposition 3.1 establishes energy preservation for the lattice permutation itself. Does this extend in any form to the full TMNO layer? Also, how do the guarantees change for the padded, non-periodic domains used in the physical benchmarks?

6 comments
AuthorsOpenReview ↗

Response part 1

We are deeply grateful to reviewer for the encouraging words on our core research direction—especially your appreciation for bringing multiparameter harmonic analysis, coupled frequency structures, and interpolation-free SL2(Z)\mathrm{SL}_2(\mathbb{Z}) modular lattice shearing into neural operator design, as well as our multi-system ablations and rotated-shock evaluations. Your careful audit of our PDE benchmark descriptions, orientation conventions, and theoretical claims pinpointed exactly where our initial exposition fell short of the mathematical standard we aimed for. We have thoroughly revised the manuscript so that every PDE setup is stated with complete physical precision and every theoretical claim aligns strictly with what the discrete architecture guarantees.

W1 & Q1–Q2. Mathematical and physical consistency of the PDE benchmarks

  1. Boussinesq / Buoyancy-driven Navier–Stokes setup (W1 & Q1, Appendix C.1): The ambiguous notation a(x)=T(x,0)a(x) = T(x,0) and T(x,0)=1T(x,0)=1 in the earlier draft has been completely replaced with the exact specification of the PDEArena conditioned 2D buoyancy-driven Navier–Stokes dataset (Gupta & Brandstetter, 2022) in Appendix C.1 (lines 923–931):

    • Input and Target: Fields are stored on a 128×128128 \times 128 grid. The operator input consists of 7 channels: the transported scalar field uu and incompressible velocity components (vx,vy)(v_x, v_y) at the first stored snapshot (t=18.0t = 18.0), the two spatial coordinate planes (x,y)(x, y), and two constant planes holding the sample's buoyancy coefficient byb_y and the stored time step Δt\Delta t. The target is the 3-channel state (u,vx,vy)(u, v_x, v_y) at the next stored snapshot.
    • Sample diversity: Rather than being constant across samples, trajectories vary across initial conditions and 19 distinct buoyancy values by∈[0.2008,0.3879]b_y \in [0.2008, 0.3879], partitioned into 608 training, 128 validation, and 128 test samples.
  2. Burgers shock orientation convention (x1+x2=constx_1 + x_2 = \mathrm{const} vs. x1=x2x_1 = x_2) and action of SL2(Z)\mathrm{SL}_2(\mathbb{Z}) shears (W1 & Q2, Section 3.1 & Appendices C.4, C.6): The reviewer caught a crucial geometric distinction, which we now state explicitly in Section 3.1 ("What the frames do", lines 229–237) and Appendix C.4 (lines 966–969):

    • Exact Fourier and physical action of G1G_1 and G2G_2: Let (Gv)(p)=v(Ap)(Gv)(p) = v(Ap) with A∈SL2(Z)A \in \mathrm{SL}_2(\mathbb{Z}). By Proposition 3.1, G−1F−1MFG=F−1M([A⊤k]N)FG^{-1}\mathcal{F}^{-1} M \mathcal{F} G = \mathcal{F}^{-1} M([A^\top k]_N)\mathcal{F}. For our default gathers G1G_1 and G2G_2, the dual frequency maps are A⊤k=(k1,k1+k2)A^\top k = (k_1, k_1 + k_2) and A⊤k=(k1+k2,k2)A^\top k = (k_1 + k_2, k_2), which promote the coupled frequency coordinate k1+k2k_1 + k_2 (reduced modulo NN) to a primary coordinate axis.
    • Distinction between the two diagonal families: In physical space, both G1G_1 and G2G_2 straighten fronts with level sets i−j=consti - j = \mathrm{const} (i.e., x1=x2x_1 = x_2), whose Fourier spectrum lies on the coupled ray k1+k2=0k_1 + k_2 = 0; neither straightens fronts with level sets i+j=consti + j = \mathrm{const} (x1+x2=constx_1 + x_2 = \mathrm{const}), whose spectrum lies on k1=k2k_1 = k_2. In the Burgers dataset of Appendix C.4, the initial condition αsin⁡(π(q1x1+q2x2)+θ)\alpha \sin(\pi(q_1 x_1 + q_2 x_2) + \theta) for q=(1,1)q = (1, 1) indeed has level sets x1+x2=constx_1 + x_2 = \mathrm{const} (the second diagonal family), whereas q=(1,−1)q = (1, -1) has level sets x1−x2=constx_1 - x_2 = \mathrm{const} (the first diagonal family).
    • Empirical verification across both diagonal families: We corrected the prose throughout to distinguish the two diagonal families, added the ablation variant TMNO-U, both diagonals in Table 2 and Table D9 (which includes SL2(Z)\mathrm{SL}_2(\mathbb{Z}) shears for both diagonal families i−j=consti - j = \mathrm{const} and i+j=consti + j = \mathrm{const}, achieving 0.00430.0043 on Burgers ID, 0.16970.1697 on held-out angles, and 0.39770.3977 on further angles), and added the Controlled Burgers Orientation Study (Appendix C.6, Table C1) evaluating 12 held-out primitive lattice directions across both diagonal families.
  3. Compressible Euler state variables (W1, Appendix C.2): We updated Appendix C.2 (lines 933–946) to specify the exact euler_multi_quadrants_openBC dataset from The Well (Ohana et al., 2024) solving the 2D compressible Euler equations with adiabatic index γ=1.13\gamma = 1.13 (40 training/validation trajectories and 10 test trajectories). The operator input is not density alone, but the full 5-channel physical state—density ρ\rho, total energy EE, pressure pp, and xx- and yy-momentum (ρu,ρv)(\rho u, \rho v)—at the first stored snapshot (box-averaged from 5122512^2 to 1282128^2), predicting the full 5-channel state at the next snapshot (Δt=0.015\Delta t = 0.015).

AuthorsOpenReview ↗

Response part 2

W2 & Q3–Q5. Theoretical scope, clarification/recall of the 936x figure, and conservation/boundary properties

  1. Which continuous properties survive in the discrete learned architecture (W2 & Q3, Appendix B.6): We added a dedicated section, Appendix B.6 ("Properties Inherited by the Discrete Layer", page 17, lines 879–912), drawing a strict boundary between mathematical guarantees of the discrete layer and empirical observations:

    • Guaranteed for all parameter values on the N×NN \times N grid:
      • Real-valued outputs: The Hermitian projection Π\Pi on the self-conjugate columns (k2=0k_2 = 0 and k2=−N/2k_2 = -N/2) ensures real inputs produce exact real outputs.

      • Exact lattice automorphism: Each sheared branch is an exact Fourier multiplier on the original periodic grid without spatial interpolation error (Proposition 3.1).

      • Hard-limit reproduction of mTHTm_{\mathrm{THT}}: As temperatures τs,τ0,τb→0\tau_s, \tau_0, \tau_b \to 0 with a=b=c=1,t=1a=b=c=1, t=1, and θs=−is1s2s3\theta_s = -i s_1 s_2 s_3, the discrete parameterization reproduces mTHTm_{\mathrm{THT}} at every grid wavenumber off the Nyquist planes (whereas at training temperatures τs=τ0=0.5,τb=0.25\tau_s=\tau_0=0.5, \tau_b=0.25, the learned symbol is a smoothed member of this family).

        • Explicit per-layer ℓ2→ℓ2\ell^2 \to \ell^2 operator norm bound (Eq. B5): Because the band and sign memberships form convex partitions of unity, each linear branch Kτ\mathcal K_\tau satisfies:

        ∥Kτ∥ℓ2→ℓ2≤∥aτ∥∞ ∥bτ∥∞ ∥cτ∥∞ ∥θτ∥∞ max⁡j∥Wτ,j∥2.\Vert \mathcal K_\tau \Vert_{\ell^2 \to \ell^2} \le \Vert a_\tau \Vert_\infty \, \Vert b_\tau \Vert_\infty \, \Vert c_\tau \Vert_\infty \, \Vert \theta_\tau \Vert_\infty \, \max_j \Vert W_{\tau, j} \Vert_2 . What is explicitly NOT guaranteed (lines 908–912):We explicitly state that we do not claim universal approximation rates for the trained network, nor that a trained symbol remains close to mTHTm_{\mathrm{THT}} or the continuous Calderón pieces, nor that the discrete band-limited network is mathematically guaranteed to preserve discontinuities or avoid Gibbs oscillations by construction; any reduction in oscillations observed in Figure 3 is empirical.

  2. Clarification and replacement of the 936x / 1.87B-parameter FNO comparison (W2 & Q4, Sections 3.2, 4.5 & Appendix B.5):

    • How the original 936x and 17.64 GFLOPs numbers were obtained (and their exact caveats): Those figures are an exact, reproducible consequence of equating spectral FLOPs between TMNO and FNO under PyTorch's FlopCounterMode (reported as Convention A in Appendix B.5, which counts a complex multiply–accumulate as a single real one, i.e., 2 FLOPs, and counts FFTs as zero). Under Convention A, TMNO (L=4,T=5,J=6,C=90L=4, T=5, J=6, C=90 on a 1282128^2 grid; 1,991,4761,991,476 parameters) measures exactly 17.64 GFLOPs17.64\text{ GFLOPs} (16.19 GFLOPs16.19\text{ GFLOPs} in spectral mixing across all N1(N2/2+1)N_1(N_2/2+1) modes). Because a truncated FNO (M=32M=32) applies its weights only on the M×MM \times M corner blocks (4LM2C24LM^2 C^2 real spectral parameters), raising FNO's channel width to C=338C = 338 to match TMNO's Convention-A FLOPs yields 18.29 GFLOPs18.29\text{ GFLOPs} and 1.872×1091.872 \times 10^9 parameters—which is 936×936\times the nominal 2.0×1062.0 \times 10^6 parameter budget (and 940×940\times TMNO's exact 1,991,4761,991,476 parameter count).
    • Why we agreed with your critique and replaced this ratio in the revised manuscript: We fully agree with the reviewer that inflating an FNO to C=338C=338 (1.87B1.87\text{B} parameters) just to match TMNO's all-mode FLOP count does not reflect a realistic efficiency advantage over a well-tuned, trainable FNO. In the revised manuscript, we have recalled the 936x headline claim and replaced it with a direct Accuracy Versus Cost analysis across realistically trainable models (Section 3.2, Section 4.5, and Appendix B.5, Table B1), reporting both Convention A (FlopCounterMode) and Convention B (full real arithmetic: 8 FLOPs per complex MAC and 5nlog⁡2n5n\log_2 n per FFT) alongside wall-clock runtimes on an NVIDIA RTX A6000 GPU (1282128^2 grid, batch size 4):
Model ArchitectureParametersGFLOPs (Conv. A)GFLOPs (Conv. B: Full + FFT)Step Time [s]150 Epochs [min]
TMNO (Full Model)1,991,4761,991,47617.6417.6470.3470.340.3420.34246.946.9
TMNO, identity frame only (C=195C=195)2,005,9352,005,93521.0121.0168.3768.370.1400.14019.419.4
FNO (M=32,C=11M=32, C=11, pm)1,984,8191,984,8190.0730.0730.1800.1800.0170.0171.91.9
FNO (M=12,C=29M=12, C=29, pm)1,945,3291,945,3290.2440.2440.5160.5160.0200.0202.02.0
AuthorsOpenReview ↗

Response part 3

As we now state explicitly in the Abstract, Section 3.2, and Section 4.5, TMNO's parameter count (1.99M1.99\text{M}) is independent of the number of retained Fourier modes, whereas its computational cost (17.64 GFLOPs17.64\text{ GFLOPs} under Convention A / 70.34 GFLOPs70.34\text{ GFLOPs} under Convention B) is substantially higher than that of parameter-matched FNOs (0.18–0.52 GFLOPs0.18\text{--}0.52\text{ GFLOPs}).

  1. Energy conservation of the full layer vs. gathers & non-periodic boundary handling (Q5, Section 3.3 & Appendix B.6):
    • Does Proposition 3.1 extend to the full TMNO layer? No. As clarified in Section 3.3 (lines 266–269) and Appendix B.6 (lines 893–902), Proposition 3.1 (ℓ2\ell^2 energy conservation / unitarity) applies strictly to the lattice gathers GτG_\tau alone (proving they permute grid nodes without numerical dissipation or interpolation error). The full TMNO block vl=GELU(GN(Kl(vl−1)+Wlvl−1+bl))v_l = \mathrm{GELU}(\mathrm{GN}(\mathcal{K}_l(v_{l-1}) + W_l v_{l-1} + b_l)) is not norm-preserving: the learned band profiles, sector table, channel mixing matrices WjW_j, pointwise skip connection, group normalization, and GELU activation all change the norm. Energy conservation holds for Kτ\mathcal{K}_\tau only if its symbol is unitary at every mode, which is not enforced; instead, the linear spectral branch satisfies the bounded ℓ2\ell^2 operator norm estimate in Eq. (B5).
    • How do guarantees change on non-periodic domains? As stated in Section 3.3 (lines 281–285) and Appendix B.6 ("Non-periodic data", lines 904–907), the lattice gathers are exact automorphisms only on square periodic grids (ZN2\mathbb{Z}_N^2) and are implemented via periodic modular wrap-around without spatial padding. Consequently, on the non-periodic benchmarks (Compressible Euler in Appendix C.2 and Helmholtz scattering in Appendix C.3), every branch couples opposite boundaries of the computational domain—a modeling approximation shared with the standard unpadded spectral baselines (FNO, TFNO, U-NO) evaluated under the same protocol.
Reviewer_LoFWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the Twisted Multiparameter Neural Operator (TMNO) for learning PDE solution operators with oblique wavefronts and multiscale structures. It introduces coupled scales associated with ξ1\xi_1, ξ2\xi_2, and ξ1+ξ2\xi_1+\xi_2 through pushforward multipliers, combining tri-axial frequency-band parameterization with six-sector phase modulation. An SL2(Z)\mathrm{SL}_2(\mathbb{Z}) integer-shear branch maps spatial features into coordinates suitable for axis-aligned convolutions. These branches are combined with channel mappings and residual connections. The intended benefit is to reduce the cost of frequency-wise parameterization through structured parameter sharing and improve representational efficiency for oblique physical features. Experiments cover Boussinesq thermal convection, Helmholtz scattering, compressible Euler shocks, and in- and out-of-distribution Burgers settings, reporting prediction accuracy, component ablations, directional generalization, cross-resolution performance, and comparisons of parameter counts and computational cost.

Critical strengths

The paper develops a relatively coherent progression from geometric motivation and theoretical construction to architectural design and numerical validation. The theoretical treatment includes detailed constructions and discussion. The evaluation goes beyond final prediction error to examine changes in orientation, the roles of individual components, cross-resolution performance, and computational cost.

Critical weaknesses
  1. Standard FNO learns a complex channel matrix R(k)R(k) at each retained frequency, without requiring frequency-wise smoothness or separability (Li et al., 2021, Sec. 4, Eq. (5)). On a finite conjugate-symmetric frequency set, setting R(k)=mTHT(k)ICR(k)=m_{\mathrm{THT}}(k)I_C implements the same phase rule on band-limited inputs, with consistent boundary conventions. Thus, the claimed intrinsic diagonal-smoothness barrier is unsupported. The abstract, introduction, and Appendix B.3 should revise this premise and the corresponding novelty claims.
Questions for authors
  1. Lemma B.1. Which regularity/restriction conditions justify (B3)–(B4)? With H={η3=η1+η2}H=\{\eta_3=\eta_1+\eta_2\}, m=1Hm=\mathbf{1}_H defines the zero multiplier on R3\mathbb{R}^3, yet restricts pointwise to 1 on HH. The conditions should cover the sign-function multipliers used.
3 comments
AuthorsOpenReview ↗

Response to weakness

We thank Reviewer LoFW for the perceptive mathematical critique and constructive observations. Both points have been thoroughly addressed in the revised manuscript.

Response to weaknesses: FNO with R(k)=mTHT(k)ICR(k) = m_{\mathrm{THT}}(k) I_C realizes the phase rule; clarification on diagonal-smoothness claims

The reviewer is entirely correct. Because a standard per-mode Cartesian spectral layer (FNO) parameterizes an independent complex weight matrix R(k)∈CC×CR(k) \in \mathbb{C}^{C \times C} at each retained wavenumber k=(k1,k2)k = (k_1, k_2) without imposing cross-mode smoothness or separability, setting R(k)=mTHT(k)ICR(k) = m_{\mathrm{THT}}(k) I_C (with consistent conventions on Σ∨\Sigma^\vee) exactly reproduces the Twisted Hilbert Transform phase rule on inputs band-limited to the retained modes. The unqualified claim that FNO faces an intrinsic expressivity barrier on fixed finite mode sets was imprecise and has been removed.

Changes in the revised manuscript:

  • Abstract & Introduction (Section 1): Revised to explicitly state that per-mode spectral operators can represent any multiplier on their retained modes (including R(k)=mTHT(k)ICR(k) = m_{\mathrm{THT}}(k)I_C), while motivating TMNO as an inductive-bias and parameter-sharing formulation that builds oblique frequency coupling into the operator rather than learning it independently mode by mode.
  • Appendix B.3: Completely rewritten as "Relation to Per-Mode Cartesian Spectral Parameterizations", contrasting TMNO and FNO along three exact dimensions: (i) Structure (shared parameters across modes via coupled dyadic bands and sign memberships vs. independent per-mode weights), (ii) Parameter count (O(1)O(1) mode-independent spectral parameters vs. O(M1M2C2)O(M_1 M_2 C^2) per layer), and (iii) Definition at unseen wavenumbers (weights defined at every frequency via continuous memberships vs. weights existing only on the retained training modes).
  • Related Work (Section 5) & Conclusion (Section 6): Updated to clarify that existing spectral parameterizations can represent oblique structure when learned from data at the training resolution, whereas TMNO builds non-Cartesian singular varieties directly into the parameterization across resolutions.

Empirical evaluation of the reviewer's construction (Appendices E.1–E.2, Table E1): We explicitly implemented and evaluated the reviewer's exact construction on the Twisted Hilbert Transform task at 2562256^2 alongside two trained full-mode FNO variants retaining all 128128 modes per axis:

Model ConfigurationValidation (2562256^2)Held-out (2562256^2)Zero-shot (5122512^2)Zero-shot (102421024^2)
FNO, initialized at the THT, then trained0.0132±0.00000.0132 \pm 0.00000.0140±0.00000.0140 \pm 0.00000.3249±0.00000.3249 \pm 0.00000.3970±0.00000.3970 \pm 0.0000
FNO, all modes (pm, trained from scratch)0.0140±0.00020.0140 \pm 0.00020.0148±0.00020.0148 \pm 0.00020.3257±0.00060.3257 \pm 0.00062.0993±0.52172.0993 \pm 0.5217
TMNO (Ours)0.0021±0.00000.0021 \pm 0.00000.0028±0.00030.0028 \pm 0.00030.0028±0.00050.0028 \pm 0.00050.0141±0.00180.0141 \pm 0.0018
TMNO, identity frame only0.0019±0.00020.0019 \pm 0.00020.0020±0.00020.0020 \pm 0.00020.0024±0.00060.0024 \pm 0.00060.0126±0.00090.0126 \pm 0.0009

Takeaway: At the training resolution (2562256^2), the untrained exact construction achieves 0.01390.0139—the lowest error this 128-mode real-FFT construction can achieve because the Fourier-truncated 2562256^2 targets carry energy in the Nyquist column k2=−128k_2 = -128, for which a 128-mode FNO has no spectral weight (Appendix E.1). Moreover, the parameter-matched all-modes FNO trained from scratch reaches 0.01480.0148, confirming the reviewer's observation that per-mode weights can learn mTHTm_{\mathrm{THT}} at the training resolution (and that the 0.46–0.640.46\text{--}0.64 errors of truncated FNOs in Table 1 stem from mode truncation to M∈{12,32}M \in \{12, 32\} rather than an inability of per-mode weights to represent the multiplier). However, under zero-shot transfer to 5122512^2 and 102421024^2, the all-modes FNOs have no weights for modes above the 2562256^2 training band and incur errors of 0.32–2.100.32\text{--}2.10, whereas TMNO's resolution-invariant parameterization transfers smoothly (0.00280.0028 at 5122512^2 and 0.01410.0141 at 102421024^2).

AuthorsOpenReview ↗

Response to questions:

Q1. Regularity and restriction conditions for the pushforward multiplier relation (Lemma B.1 and Remark B.2)

We thank the reviewer for highlighting the counterexample m=1Hm = \mathbf{1}_H. Because a general multiplier m∈L∞(R3m)m \in L^\infty(\mathbb{R}^{3m}) is defined only up to sets of Lebesgue measure zero in R3m\mathbb{R}^{3m}, naive pointwise restriction to the lower-dimensional subspace ι(R2m)=H\iota(\mathbb{R}^{2m}) = H is ill-posed without continuity off a sparse singular set, and applying Fubini's theorem to the fiber integral of TmFT_m F requires TmF∈L1(R3m)T_m F \in L^1(\mathbb{R}^{3m}). We have rewritten Lemma B.1 and added Remark B.2 (Appendix B.1, page 15) with rigorous regularity hypotheses:

  1. Admissible representatives and well-definedness (Lemma B.1(ii)): Let m∈L∞(R3m)m \in L^\infty(\mathbb{R}^{3m}) admit a representative m~\tilde{m} that is C∞C^\infty on R3m∖Z\mathbb{R}^{3m} \setminus Z for a closed singular set Z⊂R3mZ \subset \mathbb{R}^{3m} such that ι−1(Z)\iota^{-1}(Z) has Lebesgue measure zero in R2m\mathbb{R}^{2m}, where ι(ξ)=(ξ1,ξ2,ξ1+ξ2)\iota(\xi) = (\xi_1, \xi_2, \xi_1 + \xi_2). Any two representatives that are continuous off closed sets Z′Z' and Z′′Z'' agree everywhere on the open set R3m∖(Z′∪Z′′)\mathbb{R}^{3m} \setminus (Z' \cup Z''); since ι−1(Z′∪Z′′)\iota^{-1}(Z' \cup Z'') is null, the pushforward multiplier is uniquely defined almost everywhere on R2m\mathbb{R}^{2m}, independent of the choice of representative, and satisfies:

π∗m:=m~∘ι,∥π∗m∥L∞≤∥m∥L∞.\pi_{\ast} m := \tilde{m} \circ \iota, \qquad \Vert \pi_{\ast} m \Vert_{L^\infty} \le \Vert m \Vert_{L^\infty}.

  1. Resolution of the counterexample (Remark B.2): As explicitly noted in Remark B.2, for the hyperplane HH defined by η3=η1+η2\eta_3 = \eta_1 + \eta_2, the indicator function defines the zero multiplier in L∞(R3)L^\infty(\mathbb{R}^3), whose equivalence class contains the smooth representative m~≡0\tilde{m} \equiv 0 (with Z=∅Z = \emptyset), yielding:

π∗1H=0.\pi_{\ast} \mathbf{1}_H = 0.

Conversely, the raw indicator representative is discontinuous on the hyperplane Z=HZ = H, which violates our hypothesis because ι−1(H)=R2\iota^{-1}(H) = \mathbb{R}^2 does not have measure zero.

  1. Intertwining relation and unique L2L^2 extension (Lemma B.1(i)–(iii)): For every Schwartz function F∈S(R3m)F \in \mathcal{S}(\mathbb{R}^{3m}) with supp⁡(F^)\operatorname{supp}(\hat{F}) compact in R3m∖Z\mathbb{R}^{3m} \setminus Z, smoothness of m~\tilde{m} off ZZ guarantees m~F^∈Cc∞(R3m)\tilde{m}\hat{F} \in C_c^\infty(\mathbb{R}^{3m}), so TmF=F−1[m~F^]∈S(R3m)⊂L1(R3m)T_m F = \mathcal{F}^{-1}[\tilde{m}\hat{F}] \in \mathcal{S}(\mathbb{R}^{3m}) \subset L^1(\mathbb{R}^{3m}). By Lemma B.1(i), the fiber integral converges absolutely a.e. and satisfies:

F[π∗(TmF)](ξ1,ξ2)=(π∗m)(ξ1,ξ2) F[π∗F](ξ1,ξ2),∀(ξ1,ξ2)∉ι−1(Z).\mathcal{F}[\pi_{\ast}(T_m F)](\xi_1, \xi_2) = (\pi_{\ast} m)(\xi_1, \xi_2)\,\mathcal{F}[\pi_{\ast} F](\xi_1, \xi_2), \quad \forall (\xi_1, \xi_2) \notin \iota^{-1}(Z).

By Lemma B.1(iii), the family of functions F[π∗F]\mathcal{F}[\pi_{\ast} F] for such FF contains Cc∞(R2m∖ι−1(Z))C_c^\infty(\mathbb{R}^{2m} \setminus \iota^{-1}(Z)), which is dense in L2(R2m)L^2(\mathbb{R}^{2m}) since ι−1(Z)\iota^{-1}(Z) is closed and null. This uniquely determines Tπ∗m=F−1[(π∗m)⋅]FT_{\pi_{\ast} m} = \mathcal{F}^{-1}[(\pi_{\ast} m)\cdot]\mathcal{F} as a bounded linear operator on L2(R2m)L^2(\mathbb{R}^{2m}) with operator norm at most ∥m∥∞\Vert m \Vert_{\infty}.

  1. Coverage of TMNO's sign and Littlewood–Paley multipliers (Remark B.2): For the sign multiplier (m=1m=1),

m(η)=−isgn⁡(η1)sgn⁡(η2)sgn⁡(η3),m(\eta) = -i\operatorname{sgn}(\eta_1)\operatorname{sgn}(\eta_2)\operatorname{sgn}(\eta_3),

mm is locally constant (hence C∞C^\infty) off the closed coordinate-plane union ZZ where η1η2η3=0\eta_1 \eta_2 \eta_3 = 0, and its preimage ι−1(Z)=Σ∨\iota^{-1}(Z) = \Sigma^\vee is a finite union of three lines of measure zero in R2\mathbb{R}^2, so π∗m=mTHT\pi_{\ast} m = m_{\mathrm{THT}} almost everywhere. The smooth Littlewood–Paley factors ψ(2−j1η1)ψ(2−j2η2)ψ(2−j3η3)\psi(2^{-j_1}\eta_1)\psi(2^{-j_2}\eta_2)\psi(2^{-j_3}\eta_3) are smooth everywhere (Z=∅Z = \emptyset), so their products with the sign multipliers satisfy Lemma B.1 with the same singular set ZZ.

We hope we've addressed all your concerns, Since the discussion period ends in <24 hours, please let us know if you need any further clarification. If no further clarifications are needed,may we kindly request that you revise your scores upwards?

Reviewer_VKiYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces the Twisted Multiparameter Neural Operator (TMNO). TMNO models non-Cartesian multiscale singularities by combining pushforward multipliers, tri-axial Calderón filtering, and conical phase modulation. Moreover, it introduces a modular lattice shearing based on SL2(Z)SL_2(Z) for directional transport. Experiments have verified the predictive performance, out-of-distribution generalization, zero-shot super-resolution, and parameter efficiency of TMNO on multiple PDE benchmarks. Ablation experiments further demonstrated the effectiveness of modular shearing and multiscale filtering. However, the paper still has several key issues.

Critical strengths
  1. The article introduces the twisted multiparameter singular integral theory into Neural Operator Learning, providing an interesting approach for operator construction.

  2. TMNO has achieved good predictive performance on multiple PDE benchmarks. The paper also provides rotated-shock OOD evaluation, architectural ablation, zero-shot super-resolution, computational efficiency analysis, and multi-seed statistical tests.

Critical weaknesses
  1. The research motivation and core contributions are unclear. The existing FNOs can already represent different frequency structures by independently learning the spectral weights of two-dimensional Fourier modes. The specific relationship and essential differences between this spectral parameterization and the proposed method are not clear. Moreover, the current experiments are not sufficient to prove that the twisted multiparameter spectral representation itself is the key to performance improvement, rather than mainly coming from directional convolutions and lattice shearing.

  2. The paper lacks a systematic comparison with other parameter-efficient spectral architectures, so the related efficiency analysis still need further verification. Additionally, the current manuscript lacks a comparison with the recent FNO variants. Using only the rotated shock orientations of q = (1,2) and q = (2,1) for OOD is not sufficient to fully verify the geometric robustness under different directions.

Questions for authors

Why does Table D1 only report the comparison with CNO? Why are the in-distribution test results (Helmholtz) in Table 1 and Table E1 significantly inconsistent? Did the two training modes use the same configuration?

7 comments
AuthorsOpenReview ↗

Response to weaknesses:

We thank Reviewer VKiY for the constructive assessment of our motivation, experimental design, and statistical evaluation. All raised concerns have been systematically addressed in the revised manuscript.

Reponse to weaknesses1: Relationship to FNO per-mode parameterization & isolating the spectral trunk from modular lattice shearing

1. Specific relationship and essential differences vs. per-mode FNO (Section 1 & Appendix B.3): As now detailed in the Introduction (lines 038–046) and rewritten Appendix B.3, a standard FNO layer parameterizes an independent complex matrix Rϕ(k)∈CC×CR_\phi(k) \in \mathbb{C}^{C \times C} at each retained wavenumber k=(k1,k2)k = (k_1, k_2) with no smoothness or separability imposed across modes. Consequently, on any fixed finite set of retained modes, FNO can represent the same phase rule as the Twisted Hilbert Transform (Rϕ(k)=mTHT(k)ICR_\phi(k) = m_{\mathrm{THT}}(k) I_C). TMNO does not enlarge the class of multipliers representable on a fixed set of retained modes; rather, it defines a structured subfamily of per-mode multipliers that differs from FNO along three essential axes:

  • Cross-mode structural sharing: In TMNO, the spectral weight at mode kk is determined by soft dyadic band memberships uju_j and soft sign memberships (p−,p0,p+)(p_-, p_0, p_+) evaluated on k1k_1, k2k_2, and the coupled oblique coordinate k3=k1+tk2k_3 = k_1 + t k_2 (Eqs. 13–14). Modes within the same dyadic-sector region share parameters, building non-axis-aligned frequency coupling directly into the operator rather than learning it independently mode by mode.
  • Mode-independent parameter count: Standard FNO spectral parameters scale as O(M1M2C2)\mathcal{O}(M_1 M_2 C^2) per layer, whereas TMNO's parameter count depends only on the number of dyadic bins JJ and channel width CC (Eq. 16), remaining completely independent of the number of retained Fourier modes.
  • Definition at unseen wavenumbers: Standard FNO weights exist only on the retained training modes ∣ki∣≤M|k_i| \le M, whereas TMNO's continuous band and sign memberships define spectral weights at every wavenumber across all resolutions (demonstrated on 2562→5122→10242256^2 \to 512^2 \to 1024^2 zero-shot transfer in Table E1).

2. 2×22 \times 2 Factorial Ablation: Spectral Trunk vs. Lattice Shears (Appendices D.2–D.3, Tables D1–D2): To conclusively test whether TMNO's performance gain comes from the twisted multiparameter spectral representation itself or mainly from directional lattice shearing, we conducted a pre-specified 2×22 \times 2 factorial ablation crossing the spectral trunk with the Shear Setting (no shear frames vs. SL2(Z)\mathrm{SL}_2(\mathbb{Z}) lattice frames), with all four arms parameter-matched to ≈2×106\approx 2 \times 10^6 parameters and trained across 5 independent seeds on all six benchmark tasks:

Task (Primary Split)FNO trunk, no shearFNO trunk, lattice framesTMNO, identity frame onlyTMNO (Full Model)
NS buoyancy0.0267±0.0002∗0.0267 \pm 0.0002^*0.0305±0.0006∗0.0305 \pm 0.0006^*0.0168±0.0001∗0.0168 \pm 0.0001^*0.0125±0.00010.0125 \pm 0.0001
Euler0.2226±0.0090∗0.2226 \pm 0.0090^*0.2564±0.0120∗0.2564 \pm 0.0120^*0.1847±0.0043∗0.1847 \pm 0.0043^*0.1721±0.00490.1721 \pm 0.0049
Helmholtz0.0045±0.0002∗0.0045 \pm 0.0002^*0.0056±0.0006∗0.0056 \pm 0.0006^*0.0038±0.0003∗0.0038 \pm 0.0003^*0.0029±0.00010.0029 \pm 0.0001
Burgers (held-out angles)0.5987±0.0214∗0.5987 \pm 0.0214^*0.5829±0.0120∗0.5829 \pm 0.0120^*0.2384±0.04090.2384 \pm 0.04090.1749±0.02650.1749 \pm 0.0265
THT (2562256^2)0.6444±0.0101∗0.6444 \pm 0.0101^*0.5582±0.0174∗0.5582 \pm 0.0174^*0.0020±0.0002∗0.0020 \pm 0.0002^*0.0028±0.00030.0028 \pm 0.0003
Burgers-C (held-out)0.6969±0.0230∗0.6969 \pm 0.0230^*0.6064±0.0367∗0.6064 \pm 0.0367^*0.4444±0.0768∗0.4444 \pm 0.0768^*0.3013±0.01520.3013 \pm 0.0152

The five pre-specified Welch–Satterthwaite factorial contrasts (Table D2, Holm-adjusted across contrasts) establish:

Contrast Analysis (Controlled Burgers-C Held-out)Estimate [95% Welch CI]Statistical Significance
Trunk effect (TMNO vs. FNO, no shear frames)−0.2525    [−0.3464,−0.1586]-0.2525 \;\; [-0.3464, -0.1586]pHolm<0.05p_{\mathrm{Holm}} < 0.05
Trunk effect (TMNO vs. FNO, with lattice frames)−0.3051    [−0.3499,−0.2602]-0.3051 \;\; [-0.3499, -0.2602]pHolm<0.05p_{\mathrm{Holm}} < 0.05
Shear effect within TMNO trunk−0.1431    [−0.2376,−0.0486]-0.1431 \;\; [-0.2376, -0.0486]pHolm<0.05p_{\mathrm{Holm}} < 0.05
Shear effect within FNO trunk−0.0905    [−0.1367,−0.0443]-0.0905 \;\; [-0.1367, -0.0443]pHolm<0.05p_{\mathrm{Holm}} < 0.05
Trunk ×\times Shear interaction−0.0526    [−0.1474,+0.0422]-0.0526 \;\; [-0.1474, +0.0422]not significant

Conclusion: Across all six tasks, replacing the FNO trunk with the coupled TMNO spectral trunk accounts for the dominant share of the error reduction (e.g., −0.2525-0.2525 without shears and −0.3051-0.3051 with shears on Burgers-C; −0.3603-0.3603 without shears and −0.4081-0.4081 with shears on Burgers held-out angles; −0.6424-0.6424 on THT), while the SL2(Z)\mathrm{SL}_2(\mathbb{Z}) lattice shears provide a complementary secondary reduction on rotated shock tasks (−0.1431-0.1431 on Burgers-C).

AuthorsOpenReview ↗

Response to weakness 2

3. Architectural Component Ablation Hierarchy (Section 4.4, Table 2 & Appendix D.5): In Section 4.4 (Table 2), we further evaluate 10 single-component modifications around the uniform-branch baseline:

  • Modular lattice shears: Restricting to only a single lattice frame (I,G1I, G_1 or I,G2I, G_2) degrades held-out angle Burgers error from 0.16650.1665 to 0.21660.2166 and 0.19500.1950 (and significantly degrades Helmholtz error for I,G2I, G_2 to 0.00330.0033, while adding shears for the second diagonal family reaches 0.16970.1697 on held-out angles and 0.39770.3977 on further angles.
  • Coupled spectral dyadic structure: Removing cross-band Littlewood–Paley coupling incurs a statistically significant penalty on Helmholtz wave scattering and degrades Burgers ID.
  • Dyadic band resolution: J=6J=6 outperforms both overly coarse and overly fine band decompositions on held-out orientations.

Response to weaknesses 2: Parameter-efficient spectral baselines, FNO variants, compute cost, and expanded OOD orientation testing 1. Systematic comparison with parameter-efficient spectral architectures and FNO variants: In addition to standard and parameter-matched FNO, we evaluate parameter-efficient factorized and multiscale spectral variants—specifically TFNO (Tucker-factorized FNO, both parameter-matched and standard configurations), U-NO (U-shaped multiscale neural operator, both parameter-matched and standard configurations), and MWT (Multiwavelet Operator)—alongside MP-PDE, CNO, Transolver, U-Net, and CNN across our benchmark suite (Table 1, Table C1, Tables D3–D9, and Table E1).

2. Computational complexity & FLOP / runtime trade-off analysis (Section 4.5 & Appendix B.5, Table B1): We removed all speculative efficiency ratios and added a dedicated cost-accuracy analysis in Section 4.5 and Appendix B.5, reporting exact parameter counts, GFLOPs under both PyTorch's FlopCounterMode and full real-arithmetic including complex products and FFTs, per-step training time, and 150-epoch wall-clock runtime on an NVIDIA RTX A6000 GPU:

Model ArchitectureParametersGFLOPs (A: PyTorch)GFLOPs (B: Full + FFT)Step Time [s]150 Epochs [min]
TMNO (Full Model)1,991,4761,991,47617.6417.6470.3470.340.3420.34246.946.9
TMNO, identity frame only (C=195C=195)2,005,9352,005,93521.0121.0168.3768.370.1400.14019.419.4
FNO (M=32,C=11M=32, C=11, pm)1,984,8191,984,8190.0730.0730.1800.1800.0170.0171.91.9
FNO (M=12,C=29M=12, C=29, pm)1,945,3291,945,3290.2440.2440.5160.5160.0200.0202.02.0
U-Net1,962,4981,962,4981.751.751.751.750.0310.0313.43.4
CNN (pm)1,988,8481,988,84865.1265.1265.1265.120.0610.0618.18.1

As explicitly discussed in Section 4.5, TMNO trades a mode-independent parameter count for substantially higher FLOPs (70.3470.34 GFLOPs vs. 0.18–0.520.18\text{--}0.52 GFLOPs for parameter-matched FNOs) because its JJ mixing matrices act on all N1(N2/2+1)N_1(N_2/2 + 1) Fourier modes without truncation.

3. Controlled Burgers Orientation Study across 16 primitive lattice directions (Section 4.3 & Appendix C.6, Table C1): In the original Burgers dataset (Appendix C.4), changing the wavevector qq also alters the advection speed along the front normal (q1+q2q_1 + q_2), and OOD testing used only q∈{(1,2),(2,1)}q \in \{(1,2), (2,1)\}. To eliminate this nonlinearity confound and test geometric robustness across a wide range of angles, we designed the pre-registered Controlled Burgers Orientation Study using planar initial data u(x,0)=g0(πq⋅x)q^\mathbf{u}(\mathbf{x},0) = g_0(\pi q \cdot \mathbf{x})\hat{q} with rescaled amplitude αq=α∣qref∣/∣q∣\alpha_q = \alpha |q_{\mathrm{ref}}|/|q| and diffusivity Dq=Dref(∣qref∣/∣q∣)2D_q = D_{\mathrm{ref}}(|q_{\mathrm{ref}}|/|q|)^2 so that every orientation solves the exact same 1D profile evolution and differs purely geometrically. We train on the 4 axial/diagonal directions {(1,0),(0,1),(1,1),(1,−1)}\{(1,0), (0,1), (1,1), (1,-1)\} and their harmonics ∣q∣∈[1,4.24]|q| \in [1, 4.24], and evaluate zero-shot on all 12 remaining primitive lattice directions with max⁡(∣q1∣,∣q2∣)≤3\max(|q_1|, |q_2|) \le 3 (∣q∣∈[2.24,3.61]|q| \in [2.24, 3.61])

AuthorsOpenReview ↗

Response to question

Q1. Pairwise comparisons in Appendix D & consistency of Helmholtz results across tables

  1. Why the previous Table D1 only reported CNO: We have replaced that single-baseline comparison with comprehensive pairwise statistical testing across all competing models and ablation arms on the primary split of every task in Appendix D.4, reporting two-sided Welch tt-tests on the n=5n=5 seed-level means, exact 95% Welch–Satterthwaite confidence intervals, raw pp-values, and Holm-adjusted family-wise pp-values.
  2. Why Helmholtz results in Table 1 and the old Table E1 differed & whether they used the same configuration: In the initial submission, those two tables inadvertently mixed runs from different training configurations/protocols. In the revised manuscript, all models and ablation arms across the entire paper are trained under a single, unified training recipe and hyperparameter configuration(documented in Appendix C.7: AdamW with lr=10−3\text{lr}=10^{-3}, weight decay 10−410^{-4}, 8-epoch linear warmup + cosine decay over 150 epochs, batch size 4, gradient clipping 1.0, seeds 0–4). Consequently, Helmholtz test errors are now completely consistent across Table 1 (0.0029±0.00030.0029 \pm 0.0003), Table 2, and Table D1, and the revised Table E1 is now dedicated exclusively to the Twisted Hilbert Transform multi-resolution zero-shot transfer study (2562→5122→10242256^2 \to 512^2 \to 1024^2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.