acceptodds
Under review as a conference paper at ICLR 2027

Spectral Generator Neural Operator for Stable Long-Horizon PDE Rollouts

Abstract

Neural operators efficiently simulate dynamical systems by repeatedly predicting the next state. However, small errors at each step can accumulate, causing long- term predictions to drift from the true trajectory. Long-term error reflects both the propagation of earlier prediction errors and new errors introduced at each step. How to reduce these errors is therefore a critical question. We propose the Spectral Generator Neural Operator (SGNO), which combines a linear estimate of the next state with a learned correction. The linear component uses consecutive states in the training data to fit a map from the current state to the next. A neural network corrects this prediction by learning changes that the linear model does not capture, including nonlinear interactions. Both components are trained jointly toward the same next- state target. Experiments across 13 periodic PDE configurations in one to three dimensions show that SGNO accurately learns state evolution on linear tasks and reduces 100-step rollout error by 13-74% over the strongest baseline under the main comparison protocol on each nonlinear task. Longer-horizon extrapolation tests show lower late-window prediction errors on several nonlinear tasks. These results show that combining fitted propagation with learned state-dependent correction can improve long-horizon autoregressive prediction on the evaluated PDE tasks. Code is available at https://anonymous.4open.science/r/SGNO-0D23/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
PDE, neural operators, operator learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_3XiHOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the Spectral Generator Neural Operator (SGNO) for long-horizon autoregressive PDE prediction. SGNO combines a fitted linear propagation branch in Fourier space with a neural corrector that learns the dynamics not captured by the linear map. The corrector uses a spectral generator parameterization inspired by exponential integrators. Across 13 periodic PDE configurations (nine linear and four nonlinear), the authors report near-exact predictions on the linear tasks and 13–74% lower 100-step rollout error than the strongest evaluated baseline on each nonlinear task.

Critical strengths
  • The paper is clearly written, and the motivation, method, and experimental setup are easy to follow.
  • The experimental reporting is thorough and transparent, including results across multiple seeds, failed runs, and limitations of the numerical reference solutions.
  • SGNO achieves the lowest median rollout error among the main baselines on all four nonlinear tasks.
Critical weaknesses
  1. The ablations do not sufficiently isolate the contribution of the corrector design. The “Fixed identity” variant in Table 2 shows that the correction branch alone already outperforms the strongest baseline on three of the four nonlinear tasks, but the contribution of its specific architectural choices is less clear. Comparisons with reasonably tuned FNO or convolutional correctors, using the same fitted propagation branch and comparable parameter counts and computational costs, would help clarify this contribution. Further controls could compare shared versus independently parameterized weights for the two paths, and include a residual-parameterized FNO baseline to separate the correction-layer design from the residual formulation.

  2. The conditions under which SGNO is effective remain insufficiently characterized. Although the paper acknowledges the scope of its evaluation in Section 6, it remains unclear how the benefits depend on the amount of training data and the dynamics of the PDE. Table 2 shows varying gains from fitted propagation across tasks, while Table 1 shows the smallest improvement over the strongest baseline on Kolmogorov flow. These differences warrant further characterization. Experiments varying the number of training trajectories, together with comparisons across PDEs with different dynamical or spectral properties, would help identify when fitted propagation provides substantial benefits and when those benefits are limited.

Questions for authors

The main requests are in the Weaknesses section. The following clarifications would also be helpful; additional experiments are welcome but not expected for every question.

  1. Could the authors clarify how SGNO differs from the cited SSNO in practice? If feasible, a performance comparison on a shared task such as KS would also be useful.

  2. How sensitive is SGNO's performance to the corrector's capacity, and does fitted propagation continue to provide an improvement as that capacity increases? A discussion or any available evidence would be helpful.

  3. Appendix A2.2 states that FNO, CNO, and KNO “retain their recorded evaluations.” Could the authors clarify whether these baselines used the same training budget, seed protocol, and data ordering as SGNO?

8 comments
AuthorsOpenReview ↗

Response to Reviewer 3XiH (1/5)

Response to Reviewer 3XiH

We thank the Reviewer for the comments.

Summary Response:

  • We ran the controls and experiments that the reviewer suggests. FNO and CNN correctors under the same fitted propagation, a residual FNO and independent generator weights all support the corrector design (W1).
  • Fitted propagation helps most when the dynamics contain clear and repeatable modal propagation laws, and this benefit persists with five times fewer training trajectories (W2) and with a corrector that is 3.82 times larger (Q2).
  • We clarify the difference from SSNO and compare with it on KS and on eight further tasks (Q1), and we confirm that the baselines share the training protocol of SGNO (Q3).

W1. Contribution of the corrector design

The ablations do not sufficiently isolate the contribution of the corrector design. The “Fixed identity” variant in Table 2 shows that the correction branch alone already outperforms the strongest baseline on three of the four nonlinear tasks, but the contribution of its specific architectural choices is less clear. Comparisons with reasonably tuned FNO or convolutional correctors, using the same fitted propagation branch and comparable parameter counts and computational costs, would help clarify this contribution. Further controls could compare shared versus independently parameterized weights for the two paths, and include a residual-parameterized FNO baseline to separate the correction-layer design from the residual formulation.

We added the three sets of controls that the reviewer suggests, and each of them supports the corrector design. They replace the corrector under the same propagation branch, add a residual FNO, and compare shared with independent parameterization. They examine the corrector architecture, the residual formulation and the parameter efficiency, respectively. These controls are new runs under one common training protocol, with identical data within each comparison, and they are reported separately from the main-table results.

  • First, we replaced the corrector by parameter-matched FNO and CNN correctors under the same fitted propagation branch. On Kolmogorov flow, the three models use the same data and training budget and differ only in the corrector. The SGNO, FNO and CNN versions have 106,010, 104,924 and 105,889 parameters.

    Corrector under the same propagation branchParametersGMean100GMean101-200
    SGNO106,0100.4561.12
    FNO104,9240.5744.39
    CNN105,8890.49611.8
    • The values are means over five seeds.
    • SGNO is better than both replacement correctors in all five paired runs and in both windows. With the same propagation setup, data and training budget and with similar parameter counts, the corrector architecture therefore directly affects the prediction accuracy, and the SGNO corrector reaches the lower error.
    • On the computational cost, we measured the inference time of the full models on an H100 80GB with FP32 and matched grids. SGNO is close to a parameter-matched FNO in 1D and 3D, takes about 1.50 times the time of FNO in 2D, and is faster than IS-FNO(1) [1] in both 1D and 2D (Appendix A9).
  • Second, we added a residual FNO to separate the architecture from the residual formulation. Both models use the same outer identity map, so both predict the input plus a learned update. This isolates the correction architecture within a residual formulation.

    Task, mean GMean100 (5 seeds)SGNO correctorStandard residual FNO
    KS0.05170.343
    Kolmogorov0.5100.525
    • The SGNO corrector still reaches the lower error when both models use a residual formulation.
    • Adding a residual formulation to FNO alone does not give the same accuracy, which supports a contribution of the correction architecture beyond the residual parameterization.
  • Third, we compared shared with independent parameterization. We replaced the shared generator of the two spectral paths by independent parameters and kept all other settings fixed.

    Kolmogorov, mean (5 seeds)ParametersGMean100GMean101-200
    Shared generator (SGNO)106,0100.45591.121
    Independent parameterization115,1000.45181.123
    • Sharing reduces the parameter count by about 7.9%, with a 0.9% higher error over the first 100 steps and a 0.2% lower error over the second 100 steps.
    • The shared generator therefore matches the accuracy of independent parameterization within 1% in both windows with fewer parameters. This parameter efficiency motivates our use of a shared generator.

Revision location: Section 5.2 summarizes these controls. Appendix A4.5 (Tables 16 to 18) reports the corrector replacement, the residual form and the parameter sharing, and Appendix A2.3 specifies their protocols.

AuthorsOpenReview ↗

Response to Reviewer 3XiH (2/5)

W2a. Conditions under which fitted propagation helps

The conditions under which SGNO is effective remain insufficiently characterized. Although the paper acknowledges the scope of its evaluation in Section 6, it remains unclear how the benefits depend on the amount of training data and the dynamics of the PDE. Table 2 shows varying gains from fitted propagation across tasks, while Table 1 shows the smallest improvement over the strongest baseline on Kolmogorov flow. These differences warrant further characterization. Experiments varying the number of training trajectories, together with comparisons across PDEs with different dynamical or spectral properties, would help.

Fitted propagation helps the most when the state changes contain clear/repeatable modal propagation laws, and this benefit persists with five times fewer training trajectories. We first state the conditions under which effectiveness has been demonstrated and explain the differences between tasks in the original tables. We then report the two experiments on the dynamics and on the amount of training data that the reviewer suggests.

  • SGNO has the lowest median prediction error among the compared strong baselines in all 13 PDE configurations in one to three dimensions of the original manuscript, with particularly large gains on the linear tasks. These tasks use periodic boundaries, fixed grids and fixed observation intervals, and they cover propagation, diffusion, dispersion and nonlinear interactions.

  • The high accuracy on the linear tasks reflects the design goal of SGNO, which is to identify propagation laws from data and so to establish an accurate starting point for long-term prediction. For the periodic linear tasks with constant coefficients studied here, the Fourier modes evolve independently. Each step can be described by a complex multiplier, whose magnitude gives the amplitude change and whose angle gives the phase advance. SGNO fits these multipliers from adjacent training states and so learns directly how each mode evolves. Because the true update has exactly the form that the propagation branch can represent, the fit recovers the propagation laws of the modes excited by the training data, which gives a very low long-term error.

  • Nonlinear tasks also involve interactions between frequencies, so the corrector has to learn state-dependent changes. The full SGNO reaches its accuracy through joint learning of propagation and correction, and the additional benefit of fitted propagation depends on how much of the evolution these modal updates can describe accurately. This explains the varying gains in Table 2. In KdV, linear dispersion advances the phases of different frequencies at different rates. The fitted propagator estimates this pattern from adjacent observations and provides an accurate starting point, while the corrector captures the nonlinear changes. These dynamics match the representation of the propagation branch well. Nonlinear transport in Burgers is different. The local propagation speed depends on the state and the waveform steepens, and fixed modal multipliers cannot fully describe these changes. Accordingly, fitted and identity initialization give similar results after joint training in the original Burgers setting. Adding a global translation at constant speed introduces a component that modal phase advances describe accurately, and the advantage of fitted initialization then increases. The controlled experiment below tests this effect directly.

  • The state-dependent changes in Kolmogorov flow are more complex. It is a two-dimensional incompressible Navier-Stokes flow driven by periodic forcing, and its vortices evolve continually through nonlinear transport, viscous dissipation and forcing. These changes require propagation and correction to be learned jointly, and they make long-term prediction more challenging. The strongest baseline on this task in Table 1 is CNO, whose original paper uses Navier-Stokes as an important validation task and shows strong predictive performance [2]. SGNO still lowers the error from 0.494 for CNO to 0.430, a reduction of about 13.0%. The relative gain on this task should therefore be read in the light of the complex flow dynamics and the strong baseline. SGNO improves the accuracy on this task as well, and it covers a broader range of dynamics within the same framework.

AuthorsOpenReview ↗

Response to Reviewer 3XiH (3/5)

W2b. Controlled experiments

Following the reviewer's suggestion, we added two controlled experiments that examine how the composition of the dynamics and the amount of training data affect the benefit of fitted propagation.

In the control on the dynamics, we add a global translation at speed 0.25 to the same Burgers system and keep the initial states, viscosity, grid and time interval fixed. This preserves the original waveform evolution and the spectral amplitudes and adds an explicit linear transport component. Both initializations use the same data, corrector, data order and update budget. The table gives means over the same five training seeds.

Burgers, GMean100Fitted initializationIdentity initialization
No added transport0.003600.00356
Added linear transport0.002180.00801

Without added transport, the two initializations have similar errors. With added linear transport, fitted initialization lowers the error by 72.7% and improves all five seeds. Extending to ten seeds gives mean error reductions of 72.0% and 74.5% in the early and the late window, both with 10/10 paired improvements. Adding explicit linear transport to the original nonlinear evolution therefore substantially increases the advantage of fitted initialization.

In the control on the amount of data, we take the same Burgers task with added transport and reduce the training trajectories from 50 to a fixed nested subset of 10. Both initializations use the same data, data order, 10,000 updates and five training seeds, so the amount of data is compared under a fixed training budget.

Training trajectoriesFitted initializationIdentity initializationError reductionPaired improvements
100.002790.0099171.9%5/5
500.002180.0080172.7%5/5

The values are means of GMean100 over training seeds 0 to 4, with the data subset fixed. With one fifth of the training trajectories, fitted initialization still lowers the error by 71.9% and improves all five seeds. Under the tested transport dynamics, the additional benefit of fitted propagation therefore persists with less training data.

Revision location: Table 3 and Appendix A4.4 give the control on the dynamics, Appendix A8.2 (Table 26) gives the control on the training data, and Appendix A8 explains the task-dependent benefits.

Reviewer_mY7hOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Spectral Generator Neural Operator (SGNO), aiming to reduce the long-rollout error accumulation in PDE predictions. To address this, SGNO first fits a gain-constrained Fourier propagator from adjacent training states to capture linear dynamics. A neural correction network with a shared spectral generator then learns the residual nonlinear dynamics. Then they optimize both components jointly. Experiments are conducted on 13 PDE systems in 1D–3D space. Results show SGNO could achieve near-zero prediction errors on linear tasks and obvious reductions on four nonlinear tasks compared with baselines. Ablation studies show the effectiveness of least-squares initialization and each module in the SGNO layer.

Critical strengths
  1. Long-horizon autoregressive prediction remains an important and challenging problem for neural PDE surrogates. The paper is motivated by this practical issue and evaluates prediction accuracy over a long horizon.

  2. The use of constrained least-squares fitting to initialize a linear Fourier propagator is an interesting and practical design choice. It provides a data-driven estimate of the underlying amplitude and phase evolution, which is subsequently refined through joint training with a learned nonlinear correction. This may provide inspirations for other similar spectral methods.

  3. The combination of a fitted Fourier propagator and a learned correction maintains an FFT-based feed-forward architecture. The method does not require additional optimization or complex components during inference, preserving the computational scalability of spectral neural operators.

Critical weaknesses
  1. The paper emphasizes the importance of controlling error propagation in long-horizon rollouts. However, the gain constraint (|m_k|< 1) only applies to the fitted linear propagator. Although the correction network introduces bounded factors, its channel-mixing matrices, nonlinear transformations, and pointwise affine mappings are not explicitly constrained to be non-expansive. Consequently, the combined predictor may still amplify existing prediction errors. This leaves it unclear whether the improved performance arises from better control of error propagation or primarily from improved one-step prediction accuracy. This fundamentally violates the motivation and original purpose, as its error-propagation argument does not extend to the full SGNO model.

  2. Table 8 does not support the fitted linear propagator. By the mean error reported, a frozen fitted map is worse than a frozen identity on three of the four nonlinear tasks (e.g., KS: 0.0562 vs. 0.0512), and a trainable fitted start offers no consistent advantage over a trainable identity start. Apart from KdV, the differences lie within one seed SD. This suggests that the gains come from the residual corrector rather than from the fitted propagator emphasized in the authors' core contribution. If the mean and median metrics diverge, the authors should provide deeper analysis to explain.

  3. "SGNO uses width/modes/layers (11, 26, 7), (5, 10, 9), and (10, 6, 2) in 1D, 2D, and 3D." This seems like a weird choice and was heavily tuned. The paper should provide a sensitivity analysis so that the gains cannot be attributed to unfair tuning compared with other baselines.

  4. The authors should report the inference clock time compared with baselines.

Questions for authors

See weakness.

15 comments
AuthorsOpenReview ↗

Response to Reviewer mY7h (1/5)

We thank the Reviewer for the constructive comments.

Brief Response:

  • The corrector is deliberately not constrained to be non-expansive, as the true dynamics of equations such as KS amplify perturbations, and new measurements on the full model show that both error terms of Equation (2) are reduced while the amplification stays close to that of the numerical solver (W1).
  • Fitted propagation contributes wherever the dynamics contain a linear propagation component, and a new controlled experiment shows a mean error that is more than 72% lower, with improvements in all ten seeds (W2).
  • The model configurations match the parameter counts of the baselines, and we add a sensitivity analysis (W3).
  • We also report the inference time (W4).

W1a. Error propagation in the full model

The gain constraint (|m_k|< 1) only applies to the fitted linear propagator. Although the correction network introduces bounded factors, its channel-mixing matrices, nonlinear transformations, and pointwise affine mappings are not explicitly constrained to be non-expansive. Consequently, the combined predictor may still amplify existing prediction errors.

Thank you. This is a deliberate choice and a much needed central feature of the design, not a flaw.

SGNO aims to learn accurate long-term evolution while reducing accumulated prediction error, and this does not require the full model to be globally non-expansive. Not every task should be non-expansive. The true dynamics of some equations amplify perturbations. KS is an example [1], and our new measurement below confirms it. On KS the numerical solver itself amplifies perturbations, by a factor of about 1.011 per step. A model that is forced to be non-expansive everywhere cannot learn such dynamics well.

Therefore, the fitted linear propagator is constrained and we do not impose the constraint on non-linear corrector:

  • The fitted linear propagator is constrained. It scales each frequency component directly, and the same multiplier is applied at every step of a rollout, whatever the state is. A modal gain above 1 would therefore amplify the error in that mode again and again. We restrict the gains to the unit disk, so that this linear starting point cannot itself amplify existing perturbations.

  • The corrector is deliberately expansive. It captures the state changes that linear propagation cannot express, including nonlinear effects and mode coupling, and it must also represent the perturbation growth that belongs to the true dynamics.

  • When the true flow map increases the distance between two nearby states, a globally non-expansive corrector would underestimate their separation after one observation interval. While keeping this capacity, we still need to avoid excessive gains from the internal weighting and mixing, and this is the purpose of the bounded factors.

    • As described in Appendices A1.3 and A1.4, the shared generator uses eΛe^{\Lambda} and ϕ1(Λ)\phi_1(\Lambda), with values in (0,1](0,1], to bound and couple the weights of the two feature-update paths.
    • The channel-mixing matrices have spectral norm at most 1, and the weights of the source network use spectral normalization.

These choices control the internal gains and keep the capacity to learn state-dependent updates.

To test our design directly, below we add a control on KS with a non-expansive corrector. We chose KS because its dynamics combine linear instability, dissipation and nonlinear coupling, so that part of the perturbation growth belongs to the true evolution [1]. The control normalizes the entire correction branch by a sufficient upper bound of its Lipschitz constant, which makes the correction map non-expansive.

Both versions use the same network architecture and parameter count (52,812), five paired training seeds, 10,000 updates per run and the same 30 test trajectories:

KS, mean ±SD (5 seeds)Our SGNONon-expansive correctorSeeds with lower SGNO error
One-step nRMSE0.00141 ± 0.000200.0299 ± 0.00065/5
nRMSE at step 1000.261 ± 0.0531.38 ± 0.095/5
nRMSE at step 2000.852 ± 0.1092.04 ± 0.355/5
GMean1000.0576 ± 0.01340.787 ± 0.0245/5
GMean101-2000.521 ± 0.0921.69 ± 0.205/5

Our SGNO lowers the errors of the early and the late window by 92.7% and 69.1%, with improvements in all five seeds. In this control, on a task with physical perturbation growth and nonlinear coupling, the non-expansiveness constraint restricts the ability of the corrector to learn the state changes and lowers the prediction accuracy. Our choice to control the internal gains while keeping the capacity to represent true dynamical changes therefore has both a dynamical rationale and empirical support.

AuthorsOpenReview ↗

Response to Reviewer mY7h (2/5)

W1b. Does the improved performance arise from better control of error propagation or primarily from improved one-step prediction accuracy?

The gains do not come from one-step accuracy alone. We show this with two new experiments on the full model:

  • First, we compare SGNO with an FNO [2] corrector on the same Kolmogorov task. Both models use the same fitted propagation architecture and initialization procedure and are trained jointly in separate runs, with matched training data, update counts, batch size, learning rate and seeds. The one-step and the multi-step metrics come from the same 30 test trajectories, with rollouts from identical initial states.

    Kolmogorov, mean ±SD (5 seeds)Our SGNOFitted propagation + FNO corrector
    One-step nRMSE0.0296 ± 0.00160.0272 ± 0.0008
    nRMSE at step 200.235 ± 0.0070.259 ± 0.016
    nRMSE at step 500.569 ± 0.0350.710 ± 0.141

    SGNO has:

    • a slightly higher one-step error
    • a lower rollout error, by 9.3% at step 20 and 19.8% at step 50.
    • In longer rollouts the FNO corrector diverges in some seeds, while SGNO remains bounded. The long-term advantage of SGNO therefore goes beyond the ranking by one-step accuracy.
  • Second, the comparison above concerns the total error. To answer the question directly, we further measure the two terms of Equation (2) for the full model along the rollout. They are the propagated term ∥f(u~[t])−f(u[t])∥\lVert f(\tilde u^{[t]})-f(u^{[t]})\rVert and the error newly introduced at each step ∥f(u[t])−u[t+1]∥\lVert f(u^{[t]})-u^{[t+1]}\rVert, both normalized by ∥u[t+1]∥\lVert u^{[t+1]}\rVert. We use KS again. Its true dynamics amplify perturbations, so it tests whether the model amplifies errors beyond what the dynamics require. SGNO uses the same KS runs as the control above. IS-FNO(1) [3] is trained on the same data with the same settings, and its GMean100 of 0.0772 is consistent with the mean of 0.0781 in Table 7 of the appendix.

    KS, mean (5 seeds)Our SGNOIS-FNO(1)
    Newly introduced error, step 1000.001490.00170
    Propagated term, step 1000.2610.367
    Propagated term, step 2000.8521.05

    Both terms are lower for SGNO.

  • We also compared the one-step amplification of existing error of the model, ∥f(u~[t])−f(u[t])∥/∥u~[t]−u[t]∥\lVert f(\tilde u^{[t]})-f(u^{[t]})\rVert / \lVert \tilde u^{[t]}-u^{[t]}\rVert, with that of the numerical solver applied to the same perturbations. For SGNO it is 1.024 over steps 51 to 100 and 1.013 over steps 101 to 200. For the solver it is 1.011 in both ranges. The amplification of the solver itself exceeds one, which confirms that perturbation growth belongs to the true KS dynamics, as argued above. The amplification of SGNO is within about 1.3% of that of the solver, so the full model does not noticeably amplify existing errors beyond the true dynamics. On KS, SGNO introduces less error at each step and propagates it at close to the rate of the true dynamics, so less error is carried forward. Appendix A5.2 reports the complete results.

This fundamentally violates the motivation and original purpose, as its error-propagation argument does not extend to the full SGNO model.

The motivation of the paper is to address both error propagation and update accuracy, and its error analysis does apply to the full model. The paper does not argue that the full predictor should be non-expansive. Section 1 and Equation (3) make a different point. Controlling amplification alone is not sufficient, because even an update that does not amplify accumulates error when each step is inaccurate. The motivation is therefore to address both terms of Equation (2), which are error propagation and update accuracy. The error decomposition in Section 4.1, Equation (2), is written for the full predictor. It separates the propagation of existing error from the error introduced by the current update, and the joint design of propagation and correction aims to improve both terms.

The new experiments validate this at the level of the full model. The KS control shows the accuracy cost of an additional non-expansiveness constraint. The Kolmogorov comparison shows a higher one-step error together with a lower long-term error. The direct measurement of the two terms of Equation (2) shows that both are reduced, with an amplification that stays close to that of the true dynamics. The original analysis provides the design rationale, and the new experiments provide the empirical validation for the full model. Together they support the motivation of the paper, which is to improve long-term prediction and to reduce accumulated error.

Revision location. Section 5.3 and Table 4 give the matched comparison of one-step and rollout error. Appendix A4.6 (Table 19) gives the non-expansive control, and Appendices A5.1 and A5.2 (Tables 21 to 23) give the complete metrics, the protocols and the direct measurement of the two error terms.

AuthorsOpenReview ↗

Response to Reviewer mY7h (3/5)

W2a. Table 8 and the fitted propagator

Table 8 does not support the fitted linear propagator. By the mean error reported, a frozen fitted map is worse than a frozen identity on three of the four nonlinear tasks (e.g., KS: 0.0562 vs. 0.0512), and a trainable fitted start offers no consistent advantage over a trainable identity start. Apart from KdV, the differences lie within one seed SD. This suggests that the gains come from the residual corrector rather than from the fitted propagator emphasized in the authors' core contribution.

Fitted propagation does contribute, and its contribution is large wherever the dynamics contain a linear propagation component:

  • On the nine linear tasks, the fitted map alone already reaches errors of 6.20×10−66.20\times10^{-6} to 1.60×10−51.60\times10^{-5}.
  • In a new controlled Burgers experiment with added linear transport, fitted initialization lowers the error by more than 72%.
  • On the four nonlinear tasks of Table 8, the fitted and the identity start end close together because the branch that starts from the identity is also trained. We explain each point below.

i. We start with Table 8. The propagation branch that starts from the identity also changes during training. It does not remain an identity map. Training therefore narrows the difference between the two starting points, and the final accuracy depends only weakly on the initialization on these four tasks. This is why the differences in Table 8 (Table 12 of the revision) are comparable to the variation between seeds.

  • The KS means 0.0562 and 0.0512 quoted by the reviewer compare a frozen fitted map with a frozen identity. A frozen map cannot adapt to the corrector. Letting the fitted map train together with the corrector lowers the median error by 6.2%, 9.8%, 8.2% and 11.0% on KdV, KS, Burgers and Kolmogorov (Table 2). This is why our method trains both branches jointly.

  • The contribution of fitted propagation is visible where the dynamics contain a component that it can describe. On the nine linear tasks, the fitted map alone, before any neural training, already reaches a GMean100 of 6.20×10−66.20\times10^{-6} to 1.60×10−51.60\times10^{-5} (Appendix A3.2, Table 8 of the revision), below every baseline in Table 1. On these tasks the accuracy comes from fitted propagation.

ii. To test the same effect on a non-linear task, we introduced a controlled global translation into the Burgers system. We chose this setting as it keeps the original non-linear waveform evolution and adds an explicit linear transport component. The waveform continues to evolve as before while its overall position moves over time, so the model must predict both the change in shape and the change in position. A global translation corresponds to a well-defined phase advance in Fourier space, which makes it suitable for testing whether fitted propagation helps to learn this component.

The change is identical for both models. They use the same translated training/test trajectories, the same corrector architecture and initial weights, and the same parameter count, data order/update budget. The only difference is whether the propagation is initialized from the fitted map or from the identity. The fitted initialization estimates the propagation from adjacent states of the same training set, and both propagation branches are then trained jointly with their correctors. The comparison therefore tests whether fitting a starting point for the propagation from data gives an additional gain over starting from the identity on the same task. The translation is implemented by integer-grid permutations at each time frame, which preserves the waveform and the spectral amplitudes without interpolation error. Viscosity, grid and time interval are unchanged. Both groups use the direct-complex parameterization and the projection onto the unit disk after each update (Appendix A2.3).

Translated Burgers, mean (10 seeds)Identity initializationFitted initializationError reductionPaired improvements
GMean1000.007980.0022372.0%10/10
GMean101-2000.01840.0046874.5%10/10

With the same corrector/the same training budget, fitted initialization lowers the error by more than 72% in both windows, with improvements in all ten seeds. When an explicit linear transport component is present, fitted initialization therefore brings a clear gain with the same corrector architecture. This supports the complementary roles of fitted propagation and nonlinear correction.

Reviewer_9hw6OpenReview ↗
Rating
4
Confidence
1
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a new approach for neural operators to predict PDE solutions over long timescales, where errors and uncertainty tend to accumulate. The Spectral Generator Neural Operator (SGNO) learns to predict two things at once: an estimate of the next state, and a correction. The next-state estimate is based on a trained linear operator (in Fourier space), and the correction relies on a (non-linear) neural network. The new approach seems to achieve much smaller predictive error than alternative established baseline techniques.

The paper is well-written and documented, appears technically sound (although I am not able to thoroughly review and validate all claims; see below), and the contribution is substantial, as long-horizon extrapolation of PDE solutions is relevant to many fields spanning science and engineering.

Hence, I see no reason not to accept this paper. However, as I alerted the ACs very early on upon receipt of the reviewing assignment, my background does not give me the capacity the review this specific work as thoroughly as would be necessary. In particular, I lack enough knowledge of the field to evaluate to which extent the approach is novel and significant.

Critical strengths
  • The paper addresses an important topic: long-horizon PDE forecasting
  • The method seems novel, interesting, and principled
  • The paper is well written and clearly presented; the problem is introduced in a manner that makes it accessible to a broad audience (eg: "Long-term error reflects both the propagation of earlier prediction errors and new errors introduced at each step")
Critical weaknesses

I haven't identified clear weaknesses myself. However, I notice that the results focus mainly on the "well-known" set of PDEs (advection, diffusion, Burgers, Kolmogorov), which might make the findings narrow in scope.

Questions for authors

N/A

3 comments
AuthorsOpenReview ↗

Response to Reviewer 9hw6 (1/2)

We thank the Reviewer for the constructive comments. Below we answer the remark about the scope of the experiments (W1). In addition, we elaborate on the novelty and significance of our work.

W1. Scope of the experiments

I haven't identified clear weaknesses myself. However, I notice that the results focus mainly on the "well-known" set of PDEs (advection, diffusion, Burgers, Kolmogorov), which might make the findings narrow in scope.

Kindly notice we employed the classical PDE problems on which neural operators are typically studied (based on popular APEBench benchmark [1].) Our evaluation:

  • Covers 13 configurations in one to three dimensions
  • Covers transport, diffusion, dispersion and nonlinear interactions
  • Provides comparison with strong published baselines in the identical testbed proposed in the prior work
  • Shows the role of each design choice under different dynamics.
  • Provides nine of the configurations that are linear and four that are nonlinear.
  • Among non-linear ones, we included:
    • the chaotic Kuramoto-Sivashinsky (KS) equation
    • two-dimensional Navier-Stokes flow (Kolmogorov flow).

We further extended the scope in the revision. To address this concern, we added two 2D configurations that are not in the paper. Both are taken from APEBench [1] and follow exactly the protocol of the paper, with 50 training trajectories, 30 test trajectories with 200 recursive steps, a 64 × 64 grid, 5 training seeds and the same training recipe for every model.

  • Hyperdiffusion adds fourth-order dissipation. The linear tasks of the paper involve derivatives up to third order (dispersion).
  • FirstFour lets advection, diffusion, dispersion and hyperdiffusion act in one equation at the same time. The linear tasks of the paper combine at most two mechanisms (advection-diffusion).

Together they extend the set of mechanisms covered by the evaluation. We retrained all four baselines with the same recipe. The table gives the complete main comparison as the median GMean100 over 5 seeds, with the two new configurations in bold.

ConfigurationFNOCNOKNOIS-FNO(1)Our SGNO
Advection (1D)0.02360.8540.0280NF (1 of 5 seeds)2.54 × 10⁻⁶
Diffusion (1D)0.01980.2070.02741.64 × 10⁻⁴8.12 × 10⁻⁶
Dispersion (1D)0.03220.2650.02774.87 × 10⁻⁵5.01 × 10⁻⁶
Mixed dispersion (2D)0.1860.05160.1670.001226.06 × 10⁻⁶
Axis diffusion (2D)0.08040.3090.01720.003547.74 × 10⁻⁶
Rotated diffusion (2D)0.07830.3190.04610.01237.57 × 10⁻⁶
Diagonal diffusion (3D)0.1660.3310.0198not evaluated5.80 × 10⁻⁶
Unbalanced advection (3D)0.7840.1340.0432not evaluated5.77 × 10⁻⁶
Advection-diffusion (3D)0.1441.440.0171not evaluated7.11 × 10⁻⁶
Hyperdiffusion (2D)0.1850.03920.03981.50 × 10⁻³5.90 × 10⁻⁶
FirstFour (2D)0.04830.01575.90NF (2 of 5 seeds)4.39 × 10⁻⁶
KdV (1D)0.1940.4730.09080.04660.0122
KS (1D)0.5160.3600.3070.08000.0512
Burgers (1D)0.01850.2930.01960.005840.00334
Kolmogorov (2D)0.6660.4940.9760.5520.430
  • SGNO has the lowest median error on all 15 configurations.
  • On the two new configurations its error is lower than that of every baseline in each of the 5 seeds, by more than two orders of magnitude.
  • NF marks a model with non-finite errors in some seeds.
  • The revision reports the new configurations, the implementations of the baselines and the late-window errors in Appendix A11.

Complex geometries and nonperiodic boundaries are a natural next step. SGNO describes the field by Fourier modes, which suit periodic domains. For other boundary conditions the same construction can use a basis that matches them, such as sine, cosine or Chebyshev expansions. For a complex geometry it can use the eigenfunctions of the Laplacian on that domain. The propagation would then be fitted on the coefficients of this basis in the same way, and the corrector would learn the remaining changes.

The above settings require their own spatial representations and boundary treatments, so we leave them for future work. Nonetheless, we have added in the table above two new PDE tasks with different physical mechanisms.

AuthorsOpenReview ↗

Response to Reviewer 9hw6 (2/2)

C1. Novelty/significance:

Confidence: I lack enough knowledge of the field to evaluate to which extent the approach is novel/significant.

Thank you. We explain both below the novelty/significance in simple terms in hope Esteemed Reviewer can increase the confidence.

  • Why the problem matters? Many problems, e.g., weather forecasting or fluid simulation, need predictions far into the future. A model that predicts one step at a time feeds each prediction back in as its next input, so small mistakes accumulate leading to a larger drift at every step - after several steps, predictions become fully incorrect.

    We ask how to reduce the error accumulation over long prediction horizon.

  • Novelty Most neural operators, e.g., FNO [2] learn the step from one state to the next as a single black box. We thus investigated and noted two sources of the error accumulation over time:

    • errors carried over from earlier steps
    • new errors occurring at each step.

The design of SGNO follows directly from this analysis: two parts are trained together, addressing each error source:

  • The first part learns, directly from consecutive snapshots in the training data, how each wave component moves and grows or decays from one step to the next step. We limit how much it can grow, so this part cannot keep inflating earlier errors.
  • The second part is a neural network (NN) that learns what the first part misses, such as non-linear interactions, so that each step is as accurate as possible.

The second part's NN layers are built on exponential integrators, a classical method for time-dependent equations. For a linear equation with a fixed forcing term, dz/dτ=Λz+gdz/d\tau = \Lambda z + g, the exact solution over one step is z(1)=eΛz(0)+ϕ1(Λ) gz(1) = e^{\Lambda} z(0) + \phi_1(\Lambda)\, g with ϕ1(Λ)=(eΛ−1)/Λ\phi_1(\Lambda) = (e^{\Lambda} - 1)/\Lambda. Each layer uses the same form. One learned operator Λ\Lambda sets the weight of the existing features and the weight of the new non-linear features. This ties the two contributions together and keeps both weights between 0 and 1.

Experimental evidence:

We evaluated SGNO on 13 PDE configurations in one to three dimensions and compared it with widely used/recent/strong baselines:

  • FNO [2]
  • CNO [3]
  • KNO [4]
  • IS-FNO [5]
  • on shared tasks, SSNO [6].

All models use the same data and training budget:

  • 5 training seeds
  • 200-step predictions, 4x longer than the training trajectories.

In this comparison SGNO has the lowest median error on all 13 configurations:

  • On the linear tasks its error is one to four orders of magnitude lower.
  • On the 4 non-linear tasks it lowers the 100-step error by 13 to 74% relative to the strongest baseline of each task.

We also ablated the model by removing/replacing each part of the model. This includes:

  • The nonlinear source term inside each layer, whose removal raises the KdV error from 0.0122 to 173,
  • The network itself, which we replaced by FNO and CNN versions of the same size, training against freezing the fitted part, and shared against separate layer weights.
  • The full model is also far more accurate than the fitted part alone, which has an error of 0.841 on KdV against 0.0122 for the full model.

Finally, we looked at long predictions:

  • Over steps 101 to 200, SGNO has lower errors and slower error growth than the strongest baseline on the 1D nonlinear tasks.
  • At step 200 on KS, both its amplitude and its phase errors are lower, and a direct measurement shows that both the errors carried over and the errors newly introduced at each step are smaller.

References

[1] Koehler et al. APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs. NeurIPS 2024.

[2] Li et al. Fourier Neural Operator for Parametric Partial Differential Equations. ICLR 2021.

[3] Raonić et al. Convolutional Neural Operators for robust and accurate learning of PDEs. NeurIPS 2023.

[4] Xiong et al. Koopman Neural Operator as a Mesh-Free Solver of Non-Linear Partial Differential Equations. Journal of Computational Physics, 2024.

[5] Yu. An Inverse Scattering Inspired Fourier Neural Operator for Time-Dependent PDE Learning. Journal of Computational Physics, 2026.

[6] Zhang et al. Stable Spectral Neural Operator for Learning Stiff PDE Systems from Limited Data. arXiv 2512.11686, 2025.

AuthorsOpenReview ↗

Follow-up before the end of the discussion period

Dear Reviewer,

As the discussion period ends on November 18, we would like to follow up on our response and the revised paper, in which all changes are marked in blue. We hope that the broader PDE coverage with the newly added PDEs and our explanation of the novelty and significance have addressed your main concerns and increased your confidence in the work. If so, we would appreciate it if you could consider updating your confidence accordingly. If you have any further questions, we would be glad to discuss them.

Sincerely,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

General response

We thank the Area Chair and Reviewers for their comments.

The reviewers' main concerns are:

  • the scope and significance of the work
  • whether the error-propagation argument holds for the full model
  • the contribution of each of the two parts of the method
  • the fairness and cost of the comparison.

We have addressed all comments. The table below is an overview.

ReviewerConcernOur responseHow addressed?
9hw6Scope of the experiments (W1)These are the classical PDE problems of neural operator research. The 13 configurations in one to three dimensions cover transport, diffusion, dispersion and nonlinear interactions. We also add two 2D configurations, hyperdiffusion/FirstFour - SGNO has the lowest error on both. Complex geometries/nonperiodic boundaries are left as future work.Clarified+new experiments
9hw6Add novelty/significance summaryThe design follows from an analysis of two sources of long-horizon errors, aiding each other (see rebuttal). SGNO has the lowest median error on all 13 configurations.Clarified
mY7hError propagation in the full model (W1)Not every task should be non-expansive. The true KS dynamics amplify perturbations, so the gain constraint is placed on the linear propagator by design, and a non-expansive corrector raises the error in all five seeds. A direct measurement on the full model shows that both terms of Eq. (2) are lower than for IS-FNO(1), with an amplification within about 1.3% of that of the numerical solver. On Kolmogorov, SGNO has a higher one-step error and a lower rollout error.Clarified+new experiments
mY7hTable 8/fitted propagator (W2)The branch that starts from the identity is also trained, which narrows the difference between the two starts on the four nonlinear tasks. In a new controlled experiment with an explicit transport component, fitted initialization lowers the mean error by 72.0%, with improvements in all ten seeds. Per-seed results explain the different rankings by mean and median.Clarified+new experiment
mY7hModel configurations (W3)The configurations match the parameter counts of the baselines and are not tuned for each task. We added three single-factor sensitivity controls.Clarified+new experiments
mY7hInference time (W4)Measured on the same hardware. SGNO is close to a parameter-matched FNO in 1D and 3D, takes 1.50 times its time in 2D, and is faster than IS-FNO(1).Addressed by new measurements
3XiHContribution of the corrector (W1)We added 3 three suggested controls (FNO/CNN correctors) under the same fitted propagation, a residual FNO, and independent instead of shared weights. Each supports the design.New experiments
3XiHConditions where fitted propagation helps (W2)It helps most when the dynamics contain clear/repeatable modal propagation laws. With added transport on Burgers it lowers the error by 72.7%, and by 71.9% with five times fewer training trajectories.Clarified+new experiments
3XiHDifference from SSNO (Q1)SSNO integrates learned continuous-time dynamics, whereas SGNO learns the discrete state map. On KS with identical data, the mean error of SGNO is 49.7% lower, with improvements in all 5 paired seeds.Clarified+new experiments
3XiHCorrector capacity (Q2)With a corrector that is 3.82 times larger, fitted initialization still lowers the mean error by 68.4%, with improvements in all ten seeds.New experiment
3XiHBaseline training protocols (Q3)FNO, CNO, KNO and SGNO share the training data, the update budget, the seeds and the sampling protocol.Clarified

The revision includes:

  • For wider evaluation, we added two 2D PDE configurations, hyperdiffusion and FirstFour, with all 5 models and 5 seeds.
  • For the full model, we measured both terms of Eq. (2) and the one-step amplification of existing error on KS and compared them with the numerical solver, we compared one-step and rollout error on Kolmogorov under matched settings, and we trained a control with a non-expansive corrector on KS.
  • For the fitted propagation, we added a controlled transport experiment on Burgers, with 10 seeds, with 5x fewer training trajectories and with a larger corrector.
  • For the corrector, we replaced it by FNO and CNN correctors under the same fitted propagation, compared it with a residual FNO, and replaced the shared generator weights by independent ones.
  • We also added a paired comparison with SSNO on KS with identical data and comparisons on 8 further tasks, 3 single-factor architecture controls on KS, and inference time in one to three dimensions.

Reporting conventions: GMean100 is the geometric mean over steps 1 to 100 of the nRMSE averaged over the test trajectories, and GMean101-200 is the same quantity over steps 101 to 200. The main tables of the paper report the median (5 train seeds). New experiments use the mean unless states otherwise.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.