acceptodds
Under review as a conference paper at ICLR 2027

Direction, Not Bandwidth: What Limits Spectral Neural Operators

Abstract

Neural operators are adopted for accuracy that survives a change of discretization, and Fourier neural operators demonstrate it on turbulent flow, reporting that the network recovers the frequencies it discards. We show that this recovery breaks down on heterogeneous solids, where the operator damps a high frequency no more than a low one. On the elastic Green's operator of homogenization, only the model that keeps every mode is accurate: keeping sixteen modes per axis gives a relative error of 0.44, against 0.026 for keeping all of them. We show this limitation belongs to the architecture rather than to training, and remove it by changing what the spectral weights represent rather than how they are learned. First, we prove that the error of any linear mode-truncating architecture is bounded below by how much the operator varies with direction. The bound is independent of the number of modes, the width and the depth, and can be computed before training. Second, we measure it on the elastic Green's operator, its anisotropic form, and the Stokes projection. Trained nonlinear models sit at the bound, truncated ones at or above it and isotropic ones within 4%, and neither added width, nor longer training, nor a loss reweighted toward high frequencies moves them off it. On a smoothing operator, where the bound is near zero, the same models are accurate and an isotropic parameterization wins instead. The direction dependence is therefore the cause. Third, we replace the table of per-mode weights with an expansion in spherical harmonics of the wavevector direction, applied at every frequency. Six coefficients then reproduce the Stokes projection exactly, at any resolution, where no finite mode table does. Placed inside a standard homogenization solver, and initialized to reproduce that solver exactly, 81 learned coefficients halve its iteration count at a phase contrast of 100 and cut it by more than 2.5× at 1000. The analytic operator at the centre of such solvers can therefore be replaced by a learned one that reaches the same accuracy in fewer iterations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, Fourier neural operator, discretization invariance, expressivity, PDE surrogates

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HNGNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the limitations of truncated spectral parameterizations for Fourier multipliers whose symbols depend only on wavevector direction and are homogeneous of degree zero. For linear spectral networks with frequency-independent channel mixing above a cutoff, the authors derive an operator-norm lower bound based on angular variation and an ensemble error formula that accounts for the input spectrum. They propose a directional spectral layer using even angular harmonics, with coefficients shared across frequencies and a parameter count independent of grid resolution. The paper establishes exact representations at finite angular degree for the isotropic elastic Eshelby–Green symbol and the Leray projector. Two-dimensional experiments compare the method with truncated and full-band FNOs, a continuous isotropic-symbol model, and a U-Net, with additional FNO comparisons in three dimensions. The reported results show improved accuracy and parameter efficiency on the tested order-zero operators. A heat-operator control and a nonlinear homogenization task show that these advantages do not hold uniformly across targets. The authors also study resolution transfer and use a learned directional symbol in an unrolled FFT homogenization scheme, reporting fewer iterations at matched reference-field error.

Critical strengths
  1. Clear problem formulation and interpretable linear analysis. The paper identifies a concrete mismatch between direction-dependent order-zero operators and the frequency-independent matrix used above the cutoff in linear mode-truncating spectral networks. The operator-norm bound makes this limitation explicit and relates it to the angular variation of the target symbol. The paper also explains why truncation error can still decrease for smoother inputs as the cutoff increases, which helps clarify the scope of the result.

  2. A compact parameterization supported by operator structure. The directional layer incorporates the target symbols’ dependence on wavevector direction and their symmetry under sign reversal. The exact low-degree representations of the isotropic elastic Green’s symbol and the Leray projector provide a concrete mathematical justification for this design. Sharing the angular coefficients across frequencies keeps the spectral parameter count small and defines the same spectral rule on different grids.

  3. Substantial empirical gains on the tested order-zero operators. The spectral models share a common backbone and training protocol, which helps isolate the effect of the spectral parameterization. On the 2D isotropic elastic target, the directional model achieves a relative L2L^2 error of 0.0065, compared with 0.0259 for the full-band FNO, using approximately 38k and 8.4M parameters, respectively. The reported advantages across several order-zero targets in both 2D and 3D support the practical value of direction-dependent spectral weights for these tasks.

  4. Useful controls and comparisons that clarify the method’s scope. The heat-operator control shows the importance of radial dependence for a smoothing target, while the full nonlinear homogenization experiment reports better accuracy from a full-band FNO. The resolution-transfer study also shows that the full-band FNO transfers reasonably well when both grids represent the same band-limited inputs. These comparisons help identify where the proposed representation is useful and where its advantages are more limited.

Critical weaknesses
  1. The error prediction for the radial baseline is not fully justified. Theorem 4 finds the best frequency-independent matrix, but the linear version of the radial RBF baseline can vary its matrix with frequency magnitude. The paper does not show why the same prediction applies to this baseline. In particular, different radii on the finite Fourier lattice can have different angular averages, so radial freedom may improve on the best constant matrix. This observation does not determine the optimum of the actual finite RBF model, but it identifies a missing step in the argument. Until that step is addressed, the elastic predictions in Table 2 are not established as lower bounds for this baseline, and the 2.4%–4.0% below-prediction errors cannot be attributed specifically to nonlinearities. This weakens the central claim that the error of each architecture can be predicted before training.

  2. The claim that optimization has been ruled out is too strong. Section 5.2 argues that trained nonlinear truncated models remain at or above the linear predictions. However, the K=16K=16 FNO result with 1,024 training samples in Figure 2 appears below the prediction of 0.3904 in Table 2. The paper does not explain whether these results use the same input spectrum and error metric. If they do, the result would challenge the stated behavior of the nonlinear models, while leaving Theorem 4 intact because it applies only to linear models. The changes to width, training length, and loss provide useful evidence for the settings tested. Their limited gains, however, do not establish that optimization cannot contribute to the remaining error or that nonlinear models have reached an architectural floor.

  3. The solver results need clearer accuracy reporting and scope. Table 6 gives matching iteration counts and speedups, but not the actual relative field errors or physical equilibrium residuals. The residual of the 3000-iteration reference is also missing. This makes it difficult to judge how accurate the reported solutions and reference fields are. In addition, learning Γ0\Gamma^0 changes an operator inside the fixed-point equation. Initializing it at the analytic operator does not ensure that the original physical solution remains a fixed point after training, and the paper does not explain whether this property is preserved. The reported improvement after a fixed number of iterations is meaningful. A broader claim that the learned iteration can replace the original solver needs further validation, or a clearer statement that the result is limited to the prescribed number of steps.

Questions for authors
  1. Error prediction for the radial baseline. Could the authors justify the Table 2 predictions for the radial RBF baseline on the elastic targets? Theorem 4 optimizes a constant matrix WW, but the baseline’s linear version can use a radius-dependent matrix B(∣k∣)B(|k|). On the finite Fourier lattice, Ep[σ(nk)∣∣k∣]\mathbb{E}_p[\sigma(n_k)\mid |k|] need not equal the global angular mean. Could the authors show that the available radial freedom leaves the stated optimum unchanged, or provide a suitable bound or reference value that accounts for it? The comparison should use the same input distribution and error metric as the experiments.

  2. Figure 2 and the linear prediction. What is the exact error of the K=16K=16 FNO with 1,024 training samples in Figure 2? Does it use the same input spectrum and definition of relative error as the prediction of 0.3904 in Table 2? The plotted result appears below that value. If the settings match, how can this be reconciled with the claim that trained nonlinear truncated models remain at or above the linear prediction? Please also clarify how the training and loss experiments support the stronger claim that optimization-related explanations have been ruled out.

  3. Accuracy and scope of the learned solver. Does §5.6 aim to improve accuracy after a fixed number of iterations, or also to provide a replacement solver that preserves the original physical solution? If the latter, what ensures that the original solution remains a fixed point after learning Γ0\Gamma^0? Please report the relative field errors and physical equilibrium residuals for both methods at the Table 6 comparison points, together with the residual of the 3000-iteration reference, and define how the residual is computed. If the learned iteration is intended to run beyond the training horizon, please also show its error and residual behavior for additional iterations.

1 comment
AuthorsOpenReview ↗

Response to Reviewer HNGN

Thank you for a careful and constructive review. Each of your three questions led to a concrete correction in the revision.

Summary.

  • Q1. Answered by a new floor for the radial class (Prop 5).
  • Q2. The value is 0.2891, which narrows my claim.
  • Q3. The requested diagnostics are reported. The solver claim is withdrawn, and what remains holds only for a fixed number of steps.

Q1 (radial baseline). Prop 5 formalizes your argument. The best radial linear model applies, on each lattice shell ∣k∣=r|k|=r, the energy-weighted mean of the symbol over that shell. Its error is the within-shell variance, which never exceeds the constant-matrix floor and equals it when every shell samples directions uniformly. Because the radial class is closed under sums and compositions, this floor bounds linear radial networks of any depth. On the experimental inputs and metric it is 0.4998 (elastic), 0.4969 (cubic) and 0.7025 (Stokes), and the trained isotropic models sit above it (+8.1%, +7.9%, +0.7%). For comparison, computed for this response, the best fit within a single layer of the baseline's own 16-function radial basis is 0.5466 / 0.5505 / 0.7053. On the two elastic targets the trained four-layer models (0.5402 / 0.5364) lie between the radial floor and this one-layer fit. Stacking layers composes radial profiles beyond the span of one layer, and the nonlinearity may also contribute, and I cannot separate the two. The paper no longer attributes the fall below the constant-matrix value to nonlinearity, since radial freedom alone can explain it.

On prediction before training, the floors remain computable before training from the symbol, the input spectrum and the grid. They are bounds for linear models of each class and a reference, not a bound, for trained nonlinear ones. At the default budget most trained models land 0.7–18% above them, but on Stokes the K=16K{=}16 FNO is 12% below its floor (0.4367 vs 0.4972, Table 2).

Q2 (Fig. 2 at 1024 samples). The value is 0.2891 (one seed). Fig. 2 was run on the cubic-reference target (A=3.2A=3.2), not the isotropic one, and I have corrected the caption. The input spectrum and error definition match the main experiments, so the relevant floor is the cubic K=16K{=}16 floor of 0.3943 rather than the isotropic 0.3904, and the 1024-sample model is 27% below it. As you note, this leaves Theorem 4 intact, but it contradicts my statement about trained nonlinear models. The revised §5.2 reports it. A truncated FNO can fall below its floor, with more data on the cubic target, and already at 512 samples on Stokes (0.4367 vs 0.4972, Table 2).

I no longer claim that optimization has been ruled out. The evidence supports a narrower statement.

  • More width or longer training does not improve the truncated FNO (its error rises).
  • Losses designed against spectral bias improve it by at most 9%.
  • More data helps it most.
  • Removing the truncation with everything else fixed lowers the cubic-target error from 0.4653 to 0.0278, which is why I call truncation the dominant source of error.

Q3 (solver accuracy and scope). The numbers below are reported in revised Table 4 (submitted Table 6), for contrast 100 / 1000 and one seed. They use the model at the end of training, whereas the submitted table kept the best test-set checkpoint (see the general comment). This, together with a finer evaluation grid, is why the matching analytic iteration counts differ from the submitted ones (31 vs 32–33 at contrast 100, and 69 where the submitted table reported only more than 40).

  • Reference (3000 iterations) has equilibrium residual 5.5e-8 / 1.2e-5.
  • Learned, 16 iterations, has field error 0.104 / 0.433 and residual 0.121 / 0.245.
  • Analytic, 16 iterations, has field error 0.183 / 0.585 and residual 0.058 / 0.076.
  • The analytic iteration reaches the learned field error after 31 / 69 iterations, with residual 0.028 / 0.029.
  • One learned step applied to the reference changes it by 6.4% / 16%, so the original solution is not a fixed point of the learned iteration.
  • Beyond the training horizon the learned error grows, reaching 0.61 / 30 at 64 iterations.

The learned operator therefore fits the field at step 16 rather than the physics, since its equilibrium residual at that step is two to three times the analytic one. The residual (App. B) is (∑k∥σ^(k)k∥2/∑k∣k∣2∥σ^(k)∥2)1/2\big(\sum_k \|\hat\sigma(k)k\|^2 / \sum_k |k|^2\|\hat\sigma(k)\|^2\big)^{1/2} over nonzero, non-Nyquist modes. The result holds only for the prescribed number of steps. I have withdrawn the replacement claim and leave preserving the fixed point (for example, by using the learned operator as a preconditioner) to future work.

Thank you again for the time and effort you put into this review.

Reviewer_Veb9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper claims that spectral layers (such as those in FNO) truncate Fourier modes and hence exhibit limited capability on high frequency modes. Truncated FNO failed to deal with order-zero operators that depend on nothing but direction, which appear in FFT homogenization and incompressible flow solvers. This paper also demonstrates the existence of a lower bound of linear truncated FNO and the floor (the minimum error of such models for random fields). Then, it proposes a directional layer, which is based on spherical harmonics and applied to all Fourier modes. Results verified the theory and the practical usefulness of the proposal at a certain level.

Critical strengths
  • The study is well-motivated, and the approach matches the motivation. While limited to linear ones, the advantage of the proposal is theoretically provided by Theorem 2 and Corollary 3.
  • It is effective and parameter-efficient.
Critical weaknesses

The manuscript is hard to read, and I found many inconsistency across Abstract, Introduction, and the main body. At the very least, it is difficult to say this a well-structured paper, and I think a major restructuring is necessary.

(1) First of all, the manuscript might violate the format: no AI use statement. To my understanding, if the authors do not use AI, it should be disclosed.

(2) The main claim and results seem inconsistent. Abstract says the following:

We show this limitation belongs to the architecture rather than to training Trained nonlinear models sit at the bound, truncated ones at or above it and isotropic ones within 4%, and neither added width, nor longer training, nor a loss reweighted toward high frequencies moves them off it.

The main body repeats similar phrases. However, Section 5.2 shows a different trend. The error approaches the floor quickly, but then increases, suggesting an overfitting. Table 7 also shows that the results are sensitive to various settings. In particular, the spectrally reweighted loss reduces the error from 0.4423 to 0.4044 for K=16. Simple adjustments can improve the vanilla FNO, and the used setting may not be the best one. I do not doubt the negative influence of the truncation, but that is not the sole issue. Some rephrasing or clarification is needed.

(3) Abstract says "Placed inside a standard homogenization solver, and initialized to reproduce that solver exactly, 81 learned coefficients halve its iteration count at a phase contrast of 100 and cut it by more than 2.5× at 1000." However, the competitor is the simplest one, Moulinec-Suquet. For FFT homogenization, various better methods are available, such as Krylov subspace methods and fast gradient methods. So, this is overclaiming. Moreover, the detailed settings for the solver are not provided. REPRODUCIBILITY STATEMENT says Appendix A provides the data generation and training protocol, but actually it does not for this experiment.

(4) As mentioned above, enough information for reproducing results is not provided.

(5) Abstract says "The bound is independent of the number of modes, the width and the depth, and can be computed before training." However, only the bound in Theorem 2 is independent of the number of modes, where an infinitesimal mesh is assumed. The floor in Theorem 4 is a different concept depending on the number of modes.

Questions for authors

See above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Veb9

Thank you for the detailed review. The points you raise are correct, and the revision addresses each.

Presentation and structure. The abstract, introduction, contributions and conclusion were rewritten so that they make the same scoped claims as §5, namely that truncation is the dominant, not the sole, limitation. This removes the inconsistencies you found. §5 was restructured. §5.2 now compares every architecture against one table of training-free floors, §5.6 is rewritten as a limitation with its diagnostics, and the resolution study and the nonlinear-map table moved to App. C, so that the main text follows one line of argument from the theory to the floors and the comparisons. §3 gains Prop 5 and a remark on finite grids, Thm 2 and Thm 4 are distinguished throughout, and App. A adds a notation table. I also checked every figure caption against its run configuration, which found the Fig. 2 error discussed in (2).

(1) LLM usage statement. The submission omitted this statement, and the revision adds it. It states that large language models were used as assistants for writing (editing for clarity and consistency, LaTeX formatting) and for programming support (plotting and analysis scripts, and checks of reported numbers against the raw run outputs). The research questions, the theory, the experimental design and the interpretation of the results are my own, and I take full responsibility for the content of the paper.

(2) Consistency of the main claim. The two abstract sentences you quote are rewritten. The first is now restricted to linear spectral networks. The second now states that most trained models land at or above the linear floors and that nonlinearity can partly compensate for truncation, and its clause “neither … moves them off it” is replaced by “does not close the gap to the model that keeps every mode”, which the 9% improvement below does not contradict. The revised introduction and §5.2 follow the evidence you point to.

  • The test error that rises with training length is consistent with overfitting at 512 samples.
  • The reweighted loss improves the K=16K{=}16 FNO by 9% (Table 7).
  • Checking Fig. 2 also showed that it was run on the cubic target (caption corrected), and that with 1024 samples (one seed) the K=16K{=}16 FNO falls below its linear floor (0.289 vs 0.394).

Truncation is still the dominant factor. Removing it with everything else fixed (full-band FNO, same recipe) lowers the error from 0.44–0.47 to 0.020–0.028 on the three order-zero targets (Table 1), so the gap is not explained by an untuned training recipe. In the K=16K{=}16 loss sweep on the isotropic elastic target the directional layer remains at least 60× more accurate than the truncated FNO, including against the reweighted loss (0.4044 vs 0.0065, Tables 7 and 1).

(3) Solver baseline and settings. Comparing only against the basic scheme overstated the result. Conjugate gradients (Zeman et al., 2010), a Krylov method, on the same discretization reach the learned operator's error in 12 / 17 iterations at contrast 100 / 1000, and unlike the learned iteration they converge. I did not run fast gradient methods, but since conjugate gradients already match the learned iteration, the withdrawal stands either way. The learned iteration also does not preserve the physical fixed point (Table 4). I have rewritten the solver sentence in the abstract as a limitation, and §5.6 reports the result as a limitation. App. B now gives every setting of this experiment, namely microstructures, phases, reference medium, loading, basis, initialization, training, and the error and residual definitions.

(4) Reproducibility. App. B lists the data, architecture, training, loss-remedy, sweep and solver settings, and every table is generated from per-run outputs. Code and data will be released with the paper.

(5) Bound vs floor. Theorem 2 is stated on the torus, where the bound is independent of KK. On a grid of size NN the same argument gives a bound that depends on KK and NN and tends to the torus bound under refinement (Lemma 1). The revision adds this as a remark after Theorem 2. The abstract did conflate Theorem 2 with Theorem 4, whose ensemble floor depends on KK through the energy fraction ηK\eta_K above the cutoff. The abstract and §3 now state the two separately.

Thank you again for the time and effort you put in this review.

Thank you for your response.

If my understanding is correct, the revision made for "(2) Consistency of the main claim." substantially changes the claims and the story of the paper. As you acknowledged, Theorems do not directly support the performance of the actual implementation. The proposal works well problems that are suited to it (the three left problems in Figure 1 and Table 1), but it appears less performative (e.g., Table 5) or not to work (e.g., Table 4). Therefore, I cannot strongly endorse the paper.

Thank you for reading the revision and for the follow-up.

I agree that the revision narrows the claims, and I respect your assessment. There is clear room for further improvements, particularly in extending the analysis to nonlinear networks and in using the learned operator inside convergent solvers. I am continuing to work on both. Thank you for your careful engagement across both rounds.

Reviewer_kpBWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

If understood correctly, the paper investigates the limitations of Fourier Neural Operators on operators with strong directional dependence, particularly in heterogeneous elastic media. The authors argue that the performance degradation of mode-truncating architectures is fundamentally linked to anisotropy and provide a theoretical lower bound that is independent of model size or training effort. Through a series of experiments, they show that standard strategies such as increasing capacity, extending training, or modifying the loss do not overcome this limitation. To address it, they introduce a spectral parameterization based on spherical harmonics of the wavevector direction, which more naturally captures anisotropic operators and generalizes across resolutions. When integrated into a homogenization solver, the resulting learned operator matches the accuracy of the analytical formulation while significantly reducing the number of iterations required for convergence.

Critical strengths

Overall, while the paper raises an interesting and practically relevant question, the theoretical assumptions, their connection to the experimental results, and the evaluation of the proposed solver leave enough open questions that the overall contribution is not yet fully convincing. Moreover the clarity is really a burden.

Critical weaknesses
  • 1/ Theoretical justification remains incomplete. The main theoretical analysis is derived for linear FNOs, whereas the experimental results are obtained with nonlinear architectures. Although the authors acknowledge this gap, the experimental evidence does not fully support the conclusion that the observed performance limitations are primarily architectural rather than optimization-related. In particular, nonlinear activations may partially compensate for spectral truncation by generating higher-frequency components.

  • 2/ Connection between theory and experiments could be strengthened. The comparison with the proposed theoretical error floor is not entirely convincing because the assumptions underlying the bound differ from the capabilities of the isotropic baseline. The latter can adapt to the frequency magnitude and therefore belongs to a more expressive function class than the one considered in the analysis. As a result, it is not clear that the theoretical bound should apply directly, and the observed discrepancies merit a more careful discussion.

  • 3/ Evaluation of solver acceleration is somewhat limited. While the reduction in iteration count is encouraging, the paper does not report actual runtime gains. It is also unclear whether the learned iteration converges to the same solution as the original solver when run for a larger number of iterations. A comparison against state-of-the-art accelerated FFT-based solvers would provide a more complete assessment of the practical benefits of the proposed approach.

Questions for authors

I suggest that the authors involve some colleagues to have a first review to improve the clarity. See also Weaknesses for the questions raised by the paper.

1 comment
AuthorsOpenReview ↗

Response to Reviewer kpBW

Thank you for the review. Your three points identified real weaknesses, and the revision addresses each.

Summary.

  • W1. The claim about trained nonlinear models is narrowed, and the linear theorem is unchanged.
  • W2. Addressed by a new floor for the radial class (Prop 5).
  • W3. The solver claim is withdrawn, and runtime and convergence diagnostics are in Table 4.

W1 (linear theory vs nonlinear experiments). My data are consistent with the mechanism you describe, although I have not measured the output energy above the cutoff directly. With 1024 training samples the K=16K{=}16 FNO on the cubic target falls below its linear floor (0.289 vs 0.394, one seed, and Fig. 2 now draws the floor). The paper no longer claims that the limitation of nonlinear models is purely architectural, and I no longer claim that optimization has been ruled out. The evidence supports a narrower statement.

  • More width or longer training does not improve the truncated FNO (its error rises).
  • Losses designed against spectral bias improve it by at most 9%.
  • More data helps it most.
  • Removing the truncation with everything else fixed (full-band FNO, same recipe) lowers the cubic-target error from 0.4653 to 0.0278.

Truncation is therefore the dominant source of error, not the only one, and the theorem for linear networks is unchanged.

W2 (isotropic baseline is more expressive). I agree. Prop 5 now gives the floor of the radial class this baseline belongs to. On the finite lattice it is 9.7% and 11.1% below the constant-matrix value for the two elastic targets, and 0.7% below for Stokes. The trained isotropic models sit 0.7–8.1% above it, and I have removed the earlier "below the bound" discussion.

W3 (solver runtime, convergence and stronger baselines).

  • Runtime. The learned and analytic iterations cost the same per iteration (one FFT pair).
  • Convergence. The learned iteration does not converge to the original solution. The reference is not its fixed point (one-step defect 6.4% / 16% at contrast 100 / 1000), and its error grows beyond the 16-step training horizon.
  • Stronger solver. Conjugate gradients, a Krylov method, on the same discretization reach the learned error in 12 / 17 iterations. In my timing a CG iteration costs about 1.4× a basic-scheme iteration (109 vs 78 ms on a CPU batch of 64 fields at 64264^2). That makes CG comparable in wall-clock time at contrast 100 and about 1.5× slower at 1000, but CG converges.

I have withdrawn the solver claim, and §5.6 reports the result as a limitation.

Clarity. The abstract, introduction, contributions and conclusion now make the same scoped claims as §5. §5.2 is rewritten around a single table of floors, the resolution study and the nonlinear-map table moved to App. C, Thm 2 and Thm 4 are distinguished throughout, and a notation table is added (App. A). Prompted by Reviewer HNGN's question, I also checked the figure captions against the run configurations. This found that Fig. 2 had been run on the cubic target rather than the isotropic one stated in its caption, which is now corrected.

Thank you again for your time and effort for this review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and what the paper establishes

I thank the reviewers for careful and detailed reviews. The revised PDF addresses most of the points they raised. I summarize the changes here and answer each reviewer separately.

What the paper establishes after revision

  1. Theory. For linear spectral networks that truncate at a cutoff KK, the error on order-zero operators is bounded below by the angular variation of the symbol, uniformly in KK (on the torus), width and depth (Thm 2, Cor 3). For linear models of each class, an ensemble form (Thm 4) and, new in the revision, a floor for radial/isotropic models (Prop 5) give the best achievable error from the symbol, input spectrum and grid alone, without training.
  2. Representation. The directional layer represents the Leray projector and the isotropic Eshelby–Green operator exactly at finite angular degree (6 and 15 coefficients per channel pair in 3D), at any resolution (Prop 8).
  3. Empirics. On 2D order-zero targets the layer is 3–4× more accurate than a full-band FNO with 38k vs 8.4M parameters. Paired configuration by configuration with a K=16K{=}16 FNO across my width, training-length, data and loss sweeps, it is always more accurate. The ratio is smallest (11×) at 64 training samples, where the directional layer is itself data-limited, 65× at 1024 samples, where the FNO is at its best, and it is at least 60× in the K=16K{=}16 loss sweep on the isotropic elastic target, including the reweighted loss cited by Veb9 (0.4044 vs 0.0065, Tables 7 and 1).

The corrections below do not change Thm 2, Thm 4, Prop 8 or the accuracy comparisons.

Changes

  • Presentation and consistency (kpBW, Veb9). The abstract, introduction, contributions and conclusion now make the same scoped claims as §5. §5.2 now compares every architecture against one table of training-free floors, §5.6 is rewritten as a limitation with its diagnostics, and the resolution study and the nonlinear-map table moved to App. C so that the main text follows one line of argument. §3 gains Prop 5 and a remark on finite grids, and App. A adds a notation table. The two abstract sentences quoted in Veb9 (2) are rewritten. The first is now restricted to linear networks, and the second now states that most trained models land at or above the linear floors and that nonlinearity can partly compensate for truncation.
  • Floor for the isotropic baseline (HNGN Q1, kpBW W2). Its linear version can vary its matrix with ∣k∣|k|, which on a finite lattice can beat the best constant matrix. Prop 5 gives the floor of this radial class. On the experimental inputs and metric it is 0.4998 / 0.4969 / 0.7025 (elastic / cubic / Stokes), against the constant-matrix values 0.5536 / 0.5590 / 0.7071 used in the submission. The trained isotropic models sit 8.1% / 7.9% / 0.7% above the radial floor (Table 2). The submission explained the two elastic isotropic models falling below the constant-matrix value as an effect of nonlinearity. That explanation was incorrect and has been removed.
  • Trained nonlinear models (kpBW W1, Veb9 (2), HNGN Q2). The linear floor is not a bound for nonlinear models, and the revision now says so. Fig. 2 was run on the cubic target, not the isotropic one as its caption said (corrected), so its floor is 0.3943, not 0.3904. With 1024 training samples (one seed) the K=16K{=}16 FNO reaches 0.2891, below that floor. This is the compensation by nonlinearity that kpBW W1 describes. Truncation is nonetheless the dominant source of error. Removing it with everything else fixed (full-band FNO, same recipe) lowers the cubic-target error from 0.4653 to 0.0278, while data, width, training length and losses move the truncated FNO by tens of percent. Details are in the replies to kpBW, Veb9 and HNGN.
  • Solver (kpBW W3, Veb9 (3), HNGN Q3). The requested diagnostics are in Table 4. Within its 16-step training horizon the learned Γ\Gamma lowers the field error. It does not preserve the physical solution as a fixed point, has a higher equilibrium residual and diverges beyond the horizon, and conjugate gradients, a Krylov method, reach the same error in 12 / 17 iterations at contrast 100 / 1000. The replacement claim is withdrawn and §5.6 reports a limitation. Runtime is therefore given per iteration only (kpBW W3). The submitted solver table used the best test-set checkpoint, the new one the final model, and all other results were already evaluated at the end of training.
  • Other changes. The abstract and §3 separate the worst-case bound (Thm 2, independent of KK on the torus) from the ensemble floor (Thm 4, which depends on KK) (Veb9 (5)). App. B gives the settings of every experiment, including the solver, the loss remedies and the sweeps (Veb9 (3), (4)). An LLM usage statement is added (Veb9 (1)). Numbers refer to the revised PDF, where the submitted Table 6 is now Table 4 and the settings moved from App. A to App. B.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.