- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This study improves neural operators by using multiscale spatial branch and Radon–Fourier spectral branch in residual block of , which is called RSNO. The spectral branch uses orientation-dependent channel filtering and cross-orientation mixing in the same residual shell. In particular, it is realized by polar sampling and compensated adjoint gridding without a sinogram.
Critical strengths
(1) This paper addresses multiscale structure, which is an important topic in learning PDEs, and extending FNO with orientation-aware interactions is an attractive direction.
(2) The paper provides many proofs, which show the effort to give the proposed method a theoretical foundation.
Critical weaknesses
To be honest, the current manuscript is very hard to read. The Abstract is sometimes inconsistent with the real contributions or explanations in the main body. The definitions in earlier sections are overridden in later sections. Although I spent a fair amount of time to read it, I am not very confident that I understood it correctly.
(1) angular mixing The contribution of the angular mixing in the spectral branch is unclear. Comparing Full RSNO (e.g., 0.4426 ± 0.1870 for Darcy) and No angular mixing (0.4455 ± 0.2033) in Table 5, the improvement by the angular mixing is limited and almost negligible, given the standard deviation in Table 4. So, Abstract and Introduction are somewhat overclaiming the contribution. The improvement by the spectral branch seem significant. No visualizations of angular map are provided. They might help understanding its contribution.
(2) Baseline implementations It is unclear why No Radon spectral branch in Table 5 is much better than multiscale baselines (e.g., U-NO, MgNO) in Table 1 on Navier-Stokes. These architectures are all multiscale. So, the difference might come from some difference in training procedures, and comparisons might have been unfair. Detailed experimental settings and/or computational budgets are expected to be disclosed. I found that reported results of baselines (e.g., GNOT, RNO) tend to be worse than those provided in the paper of RNO. It is also unclear why the training epochs for FNO, U-NO, F-FNO are truncated, as mentioned in Table 10.
(3) Inconsistency between Theorems/Contributions and Implementations Abstract says "show that different angular actions range from standard Fourier multipliers to nonlocal Fourier-integral structures" and "principled and analyzable global component". However, in the real implementation, Appendix B says "only indices 1, . . . , Nr − 1 are passed to the mixer; the spectral-branch output at DC and beyond the cutoff is zero." Then, Theorems A.x say nothing. I found other explanation not to hold for the actual implementations, such as "the linear branch does not move information between different radial frequencies."
(4) Details of implementations. cross-orientation mixing requires the spatial origin of the orientation, but it is not mentioned clearly. "When a Radon spectral branch is applied, a latent channel is extended by zero outside Ω, yielding a function on Rd". Then, the periodicity of Navier-Stokes is not guaranteed, whicle the spatial branch did. In contrast, for Burgers equation, "Circular convolutional padding is applied along both axes as an architectural convention," but the time axis should not be periodic.
Questions for authors
See above.
9 comments
Response to Reviewer PoQw (part II)
A1. On Darcy, the seed-0 test error was 0.4455% without angular mixing and 0.4426% with it. We agree that this is a small increment. We also tested a task that explicitly requires a directional transfer. Each example is a random sum of 24 Fourier modes; the target rotates each wavevector by 90° while preserving its magnitude and coefficient. Across paired seeds 0, 1, and 2, turning mixing on reduced mean test error from 4.4845% to 0.6382%. The paired reduction was 3.8463 ± 0.0623 percentage points (mean ± sample SD across seeds). The enabled model has 161,090 parameters versus 119,618. As a separate check, we zeroed only the off-diagonal angular weights in the three trained models: validation error rose from 0.60–0.66% to 100.20–100.69%. This establishes that the fitted models used those connections for the rotation. The individual seed results are given in our response to Reviewer uZaQ.
Figure 7 in the revised appendix shows the learned angular mixing weights for Darcy flow and the synthetic 90° rotation task on the same color scale. On the rotation task, a pronounced band connects input and output orientations separated by 90°, as prescribed by the target map. The Darcy panel shows no comparable band. Each cell shows the root-mean-square magnitude of the learned complex update from input angle to output angle . On the rotation task a band follows , the Darcy weights are more dispersed.
A2. On NS, the methods use the same 1,000 training trajectories and 200 test trajectories and are scored on the same ten-frame forecast with their final checkpoint. The spatial-only RSNO in Table 5 retains dilated local convolutions and learned coarse-grid branches. U-NO uses a multilevel operator encoder–decoder, and MgNO uses a multigrid V-cycle. “Multiscale” describes a feature shared by all three, not an identical backbone. The archived NS runs also show substantial differences in optimization budget.
NS model (seed 0) Trainable parameters Batch size Epochs Optimizer updates Training time (min) Test relative L² (%) RSNO-H10, spectral branch disabled 373,105 8 500 62,500 192.3 7.3771 U-NO 30,478,033 16 500 31,500 68.4 13.7822 MgNO 2,922,508 50 500 10,000 98.6 12.5206 The RSNO ablation used teacher forcing, a growing training horizon, and a relative L² loss over that horizon. U-NO and MgNO used prediction feedback over ten steps and summed per-step relative errors. The evaluation is shared, and the measured gap combines architecture and training-protocol effects. To faithfully reproduce the results reported in the original papers, we used each baseline’s original training protocol without modification. For all methods, the anonymous repository provides the implementations, detailed experimental settings, and computational budgets.
The original GNOT paper (Section 4.1 and Table 1) identifies Darcy and time-dependent Navier–Stokes as FNO benchmarks, but it does not specify the spatial resolution used for its Darcy result or the viscosity used for its Navier–Stokes result. The errors reported for the other baselines in its Table 1 are also insufficient to determine these settings. To evaluate GNOT on the same samples as the other methods in our manuscript, we retrained it on our datasets, using an 85 × 85 grid for Darcy and a viscosity of for Navier–Stokes. Differences in the experimental settings may contribute to this gap, although the published information does not establish its cause.
The RNO paper (Section 4 and Table 1) describes a common training protocol for the competing methods, but it does not document a separate GNOT run for either benchmark. Several baseline entries in its Table 1 closely match those in the Transolver paper. In particular, both papers report exactly 1.05% on Darcy and 13.8% on Navier–Stokes for GNOT. Transolver explains in Appendix B.3 that it used previously published results for benchmarks already evaluated in the GNOT paper. The agreement across the three papers therefore does not establish that the reported GNOT values were obtained on the same dataset configurations and sample splits used in our manuscript. For a discussion of the discrepancy between our RNO reproduction and the published RNO results, please see our response to Reviewer eByq.
Response to Reviewer PoQw (part III)
On Burgers, the stopping criterion for FNO, U-NO, and F-FNO was a fixed budget of 50,000 optimizer updates per model. We chose this budget to give the three one-step models the same number of updates. Each of the 9,000 training trajectories provides 40 adjacent-frame pairs, so one epoch contains 360,000 pairs and, at batch size 64, requires 5,625 updates. All three runs therefore completed eight full epochs and 5,000 updates in the ninth, which is what the 9* entries in Table 10 denote. The stopping rule was set before training and was independent of test error. RSNO predicts the full trajectory from the initial condition and completed 141,000 updates over 500 epochs.
A3. In the linear spectral layer of an FNO, the learned matrix mixes channels independently at each Fourier frequency: \widehat{v}{\mathrm{out}}(\xi)=W(\xi)\widehat{v}{\mathrm{in}}(\xi). Its continuous analogue is a matrix-valued Fourier multiplier. Proposition A.4 recovers precisely this case when angular mixing is disabled. RSNO extends this parameterization by writing and allowing the output at to depend on inputs at for other directions . For a smooth angular kernel satisfying the symbol assumptions of Theorem A.6, this interaction can give rise to a nonlocal Fourier-integral structure. The theorem identifies the corresponding canonical relation: away from the spatial origin, it can connect distinct points and with and radial covectors of the same signed magnitude.
Appendix B describes how this construction is discretized. At , every polar angle denotes the same Fourier coefficient, so there is no distinct orientation to mix. The discrete reconstruction ramp also vanishes there. We therefore omit DC from the angular mixer, avoiding an ill-defined angular parameterization at the origin. The implementation retains radial indices and sets the polar output to zero beyond the cutoff. Omitting DC is consistent with the treatment of the frequency origin in Theorem A.6, which assumes that the angular kernel vanishes in a neighborhood of .
The statement that “the linear branch does not move information between different radial frequencies” refers to the whole-space polar representation. For every , the output on the shell of radius depends only on input coefficients on that same shell. The discrete mixer preserves this property exactly at the level of polar indices: an output at index uses only inputs at index , even when their angles differ.
Response to Reviewer PoQw (part I)
Thank you for the careful reading and constructive comments on the presentation. We recognize that the original manuscript did not sufficiently distinguish the central architectural idea from its mathematical analysis and numerical realization. We have clarified these distinctions in the revised manuscript.
The central idea of RSNO is straightforward. The Fourier Neural Operator (FNO) provides a familiar starting point for learning PDE solution operators. In a standard FNO spectral layer, each retained Fourier mode has a learned matrix that mixes channels at that frequency. The layer can therefore change the amplitude and phase of an existing mode, but it cannot directly transfer its coefficient to a different frequency. Some operators require precisely this transfer: a rotation, for example, moves a Fourier coefficient from one direction to another while preserving its frequency magnitude. Learning an independent coupling between every pair of retained frequencies would make the parameter count grow quadratically with the number of modes. RSNO instead writes a frequency in polar form, ξ=rθ, and allows coefficients at different orientations θ to interact at a fixed radius r. The Fourier-slice theorem gives this construction its Radon interpretation: the Fourier transform of a Radon projection at orientation θ and detector frequency r gives the coefficient v(rθ). This is the central design idea behind RSNO. Our main representation theorem makes it precise for the continuous linear branch: at each radius, the output is obtained by applying a learned angular and channel map to the input coefficients on that same shell.
For a two-dimensional field, the Radon transform records its integrals along lines. The unit normal θ specifies the direction perpendicular to a line, and the signed offset s specifies its position. The Fourier-slice theorem gives Fs[Rv](θ,r)=v(rθ). At fixed r, the coefficients therefore describe different directions at the same spatial frequency magnitude. An orientation-dependent multiplier changes the amplitude or phase of the coefficient already present in a direction. Cross-orientation mixing additionally allows the output in that direction to depend on coefficients in other directions on the same shell.
The rotation example in Appendix A (after THeorem A.5) illustrates the connection to wavefront transport. Under the map v(x)↦v(QTx), a wave at k becomes a wave at Qk. For a nonsmooth field, the same map carries a wavefront at position and covector (x0,ξ0) to (Qx0,Qξ0). The angular action specifies the transport; the Radon–Fourier coordinates make it possible to express it. The theorem concerns specified continuous angular actions under high-frequency assumptions. Our trained models have a finite frequency cutoff. The rotation experiment below uses smooth inputs. Its measured quantity is directional transfer at resolved frequencies.
We have also clarified the scope of the theoretical results. The exact shellwise representation, Sobolev estimates, and wavefront characterizations concern the continuous linear spectral branch under their stated assumptions. They are not claimed as exact properties of the complete finite-dimensional nonlinear network. Sections 3.5 and Appendix A.6 now make this distinction explicit.
Finally, the revised experiments distinguish the contribution of the Radon spectral branch from the additional angular mixing. We acknowledge that angular mixing yields relatively small improvements on the main PDE benchmarks, while the new rotation experiment demonstrates its effectiveness for a task requiring directional frequency transfer.