acceptodds
Under review as a conference paper at ICLR 2027

CoMNO: Commutator Memory Neural Operators for Long-Horizon PDE Prediction

Abstract

Solving nonlinear partial differential equations (PDEs) over long time horizons is computationally expensive using traditional numerical methods. While neural operators provide an efficient alternative, they suffer from the accumulation of approximation errors along the predicted trajectory, typically caused by the failure to capture fine-scale details. In this work, we design a Commutator Memory Neural Operator that identifies fine-scale dynamical details by commuting two operations and leverages these details to enhance the neural operator's capability in capturing fine-scale features over long time horizons. Specifically, we identify a performance gap between two operations: downsampling the high-resolution field before rollout prediction versus performing rollout prediction before downsampling. We observe that fine-scale dynamical details are often encoded in this gap. By commuting these two operations, the fine-scale details removed during downsampling can be captured. Furthermore, we employ a memory module to store these fine-scale features along the trajectory and retrieve them during rollout, generating spatially adaptive corrections that enhance long-horizon fidelity. Empirical results across five systems demonstrate that CoMNO improves prediction accuracy and long-horizon rollout stability. The complete model implementation is provided in the supplementary material.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, Long-Horizon PDE Forecasting, Scientific Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_JaHDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoMNO, which constructs closure evidence by comparing prediction-before-downsampling with downsampling-before-prediction. A recurrent memory preserves this evidence, while spatially routed experts produce additive corrections to the base prediction. Experiments across five PDE systems report improved long-horizon accuracy and physical fidelity.

Critical strengths
  1. The method targets the persistent influence of unresolved spatial structures on long-horizon prediction.
  2. The commutator signal, recurrent memory, and spatially adaptive experts constitute a coherent correction pipeline, and the ablations indicate that each component contributes to rollout performance.
  3. Experiments span five PDE systems, spatial coarsening, high-frequency Fourier errors, module ablations, and frozen-checkpoint interventions.
Critical weaknesses
  1. The observable commutator is obtained by further coarsening xtx_t, whereas the target closure concerns scales that are already lost when moving from utu_t to xtx_t. In the absence of a comparison against an oracle high-resolution closure residual, the experiments establish the predictive utility of this signal, but not its interpretation as unresolved fine-scale influence.
  2. Although the neural-operator baselines are broad, no explicit learned closure model is included, which leaves the specific advantage of commutator-based closure over alternative closure formulations unclear.
  3. The retrained "w/o Operator Commutator" variant removes both the evidence and discrepancy losses, whereas the frozen-checkpoint intervention suppresses the evidence only at inference. Consequently, the effects of the online signal and of its training-time regularization are not fully disentangled.
Questions for authors
  1. How does the observable commutator, obtained by further coarsening xtx_t, relate to the observation-grid closure residual induced by information already lost when moving from utu_t to xtx_t? Is there any direct comparison against, or correlation with, an oracle residual derived from high-resolution dynamics?
  2. Could the authors disentangle the respective contributions of the online commutator evidence and of LcommL_{\mathrm{comm}}? The current retrained ablation removes both.
  3. What evidence substantiates the advantage of commutator-based closure over existing learned closure formulations? Are comparisons with explicit closure models available under a matched protocol?
Reviewer_hnFHOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces COMNO, a framework try to study the problem error accumulation in autoregressive modeling of the PDE dynamics. Specifically, they identify the empirical difference between two rollout mechanisms: (1) downscale before rollout; and (2) rollout before downscale. To this end, they proposed a memory component trying to leverage the difference between these two mechanism and improve the performance. Emprirical study are carried on classical datasets including burgers, navier stokes, and etc.

Critical strengths

The papers study the important problem of long-term rollout, and carry out empirical study to evaluate their idea.

Critical weaknesses

My real concern is that the whole idea seems to lack reserach foundations. First, the downsampling and its relationship to the rollout seems to be well established, while I am not sure what's the real findings the authors bring from the project. And secondly, the memoery is no longer a new component from the research perspective, from the LSTM to latent modeling, what's the real novel impact this project brings is not clear for me.

And more specifically, I have following problems:

(1) The online commutator is a theoretically motivated proxy for correction needs at a further-coarsened resolution. However, showing the overall performance is in general better does not mean that they are capable of fixing the unresolved physical contribution at the fine scaled resolution.

(2) The experiments show that the components help within CoMNO. They do not fully establish that this particular combination is better than a comparably equipped prior method.

(3) Regarding the notation, reference dynamics and learned dynamics should be separated.

Questions for authors

Please see the weakness.

Reviewer_RRdHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

CoMNO aims to improve long-horizon autoregressive PDE prediction through cross-resolution evidence and recurrent memory. It compares two paths using a shared predictor: prediction followed by downsampling, and downsampling followed by prediction. Differences in the predicted states and memory outputs condition a ConvGRU memory module and spatially adaptive closure experts, which correct the base neural operator’s prediction. The evaluation covers three APEBench systems and two RealPDEBench systems, with six neural baselines, extended rollouts, spatial coarsening, physical diagnostics, and component ablations. The reported results suggest useful improvements, although inconsistencies in the statistical reporting and unresolved questions about the proposed mechanism limit my confidence in the conclusions.

Critical strengths
  1. The architecture is well motivated, and the baseline selection is relevant.

The model connects cross-resolution discrepancies, temporal memory, and spatial corrections in a coherent prediction pipeline. Including MemNO and PDE-Refiner is particularly useful because they address two closely related directions: memory and iterative correction. Table 1 reports a reduction in Burgers relative L2 error from 0.0969 for F-FNO to 0.0568 for CoMNO. On Cylinder, the best baseline obtains 0.1242, compared with 0.1151 for CoMNO. These results suggest that the combination deserves further investigation.

  1. The paper examines how the signal affects a trained model.

The authors supplement retrained component ablations with interventions that disable or rescale the commutator signal while keeping model parameters fixed. In Table 2, restoring the signal on Cylinder reduces one-step error by 0.0003, while the late-horizon reduction reaches 0.0679. This is useful evidence that its effect develops through recurrent prediction. Tables 34–35 provide complementary component-removal results.

Critical weaknesses
  1. Some reported statistics are mathematically incompatible with the stated protocol, and the evaluation definitions need reconciliation.

Table 1/Table 5 reports a Mean Corr. of 0.9946 ± 0.0193 for F-FNO on Burgers, and Table 7 reports 0.9925 ± 0.0273 for CoMNO on KS. The metric is defined as an average of Pearson correlations, Appendix D specifies five random seeds, and the tables describe the entries as means and standard deviations across runs.

For five observations bounded above by 1, the sample standard deviation cannot exceed sqrt(5) × (1 − mean). The corresponding upper bounds are approximately 0.0121 and 0.0168, both below the reported standard deviations. Rounding cannot explain either discrepancy. At least one of the values, aggregation descriptions, or table labels therefore needs correction. I would need the individual seed results and aggregation code to resolve this concern.

There are additional reporting ambiguities. Tables 29, 30, and 33 describe StepRes as a normalized governing-operator magnitude, whereas Eq. (11) and Appendix C.5 define a discrepancy between the next prediction and a reference time-stepper applied to the preceding predicted state. These measure different quantities.

The relationship between the main and extended-rollout results is also unclear. For example, F-FNO’s Burgers relative L2 error is 0.0969 in the main table, but 0.4601 at 100 steps and 1.3937 at 197 steps in Table 19. Different evaluation windows could explain this, but the manuscript needs to specify the windows, starting points, and aggregation rules associated with each table.

  1. The evidence does not yet isolate the proposed mechanism or establish its physical interpretation.

Equation (3) compares a learned predictor across resolutions while downsampling both the state and memory. The resulting discrepancy can contain physical scale effects, resolution-dependent model error, and information loss from memory compression. Appendix Eq. (13) relates this discrepancy to a correction needed on the further-coarsened grid. It does not establish that the processed signal represents the unresolved physical correction needed on the observation grid.

The ablations leave several explanations entangled. Appendix C.6 states that removing the Commutator also removes its discrepancy penalties. Removing the Experts disables the entire adaptive correction branch and associated regularizers. These experiments support the usefulness of the component packages, but do not separate evidence conditioning from regularization or establish the benefit of expert routing over a single correction network. A comparable-compute multiscale-feature control would be especially informative.

The novelty discussion also draws too broad a distinction from prior work. MemNO [1] already studies the value of memory under low-resolution observations. Parish and Duraisamy [2] combine two-level coarse-graining through the Germano identity with finite-memory closure modeling. Both are cited here, yet the statement on page 3 that previous memory methods operate “without measuring scale mismatch” needs qualification. CoMNO’s specific combination of learned-predictor discrepancies, ConvGRU memory, and spatial experts may be useful; its independent advantage remains insufficiently established.

References

[1] Buitrago Ruiz et al. On the Benefits of Memory for Modeling Time-Dependent PDEs. ICLR, 2025.

Questions for authors
  1. Could you provide the five individual Mean Corr. results behind the two inconsistent entries?

Please include the standard-deviation calculation and clarify whether the remaining tables use the same aggregation procedure.

  1. Please clarify the evaluation pipeline.

What forecast windows, starting points, and aggregation units are used for the main tables, Tables 19–28, and the signal-intervention experiments? Which StepRes definition is implemented?

Appendix C.3 also states that the input histories are coarsened while future targets remain on the original grid. How are predictions returned to that grid, and which grid is used for subsequent rollout steps? Please explain how this is handled for the baselines as well.

  1. What evidence distinguishes the commutator mechanism from generic multiscale conditioning?

Do you have separate signal-only and regularization-only results, or a comparable-budget control that replaces the commutator with ordinary multiscale features? A direct comparison with a reference closure term would also help assess the physical interpretation. These results could materially change my assessment.

  1. How does the proposed approach advance beyond existing dynamic multiscale memory closures?

Please discuss the relationship to Parish and Duraisamy [2] more specifically and reconsider the broad characterization of prior memory methods on page 3.

  1. How does CoMNO compare with a classical numerical solver in runtime?

Tables 36–40 compare neural models only. Have you measured runtime against the reference numerical solver over the same physical time horizon at comparable accuracy? This would help assess whether CoMNO offers a practical speed advantage.

References

[2] Parish and Duraisamy. A Dynamic Subgrid Scale Model for Large Eddy Simulations Based on the Mori–Zwanzig Formalism. Journal of Computational Physics, 2017.

Reviewer_3TcUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CoMNO uses an online commutator between evolve-then-downsample and downsample-then-evolve paths to identify cross-resolution discrepancies. It stores these signals in a recurrent closure memory and uses spatially routed experts to produce corrections during autoregressive neural-operator rollouts. The idea is well motivated, and the paper reports strong results on five PDE systems.

Critical strengths
  1. The commutator provides a concrete, interpretable signal for unresolved-scale effects.
  2. The frozen-checkpoint interventions and module ablations support a functional role for the commutator, memory, and closure experts.
  3. The appendix contains broad evaluations. In particular, the gains on KS and Kolmogorov are substantial; the gains on FSI are smaller but consistent.
Critical weaknesses
  1. Selective main-text reporting. The abstract and contributions claim improvements across five systems, but the main comparison includes only Burgers and Cylinder. The complete results are available in the appendix, so this is primarily a presentation issue, but the main text should include at least a compact five-system summary and clearly report where gains are small.

  2. Novelty relative to memory-augmented operators. The central distinction from MemNO is the use of online commutator evidence to condition a recurrent closure and spatial expert routing. This is potentially meaningful, but the paper does not establish how much of the improvement comes specifically from the commutator rather than from the larger memory-plus-expert architecture. The novelty claim should be positioned more carefully and supported by a stronger memory-only comparison.

  3. Accuracy versus inference cost. CoMNO is competitive on the APEBench timings, but it is considerably slower on the realistic flow tasks: about 937 ms/window on Cylinder versus 398 ms for F-FNO and 168 ms for Poseidon, and about 1672 ms on FSI versus 142 ms for F-FNO. On FSI, the accuracy improvement over MemNO is modest. The paper should discuss this tradeoff more directly and report parameter/FLOP or matched-latency comparisons.

  4. Missing controlled sensitivity studies. The paper does not evaluate (i) a commutator-free model with comparable memory and parameter count, or (ii) the effect of recomputing the commutator every step versus every two steps. These experiments are important for attributing the gains and assessing whether the extra provisional rollouts are necessary.

  5. Coarsening limitations. CoMNO improves 12 of 18 summarized coarsening comparisons, while the remaining changes are small (up to 3.5%). This is encouraging but not a universal robustness result; the limitations should be stated explicitly in the main text.

Questions for authors
  1. What remains of the gains under matched parameter count, FLOPs, or inference latency?
  2. How does CoMNO compare with a same-memory architecture in which the commutator path is removed?
  3. How does accuracy and cost change when the commutator is recomputed every step, every two steps, or less frequently?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.