acceptodds
Withdrawn from ICLR 2027

DESSNO: Enhancing Neural Operators for Multiscale PDE Systems with DCT-based Explicit Scale Separation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural operators have emerged as efficient surrogates for partial differential equation (PDE) simulations, yet accurately resolving multiscale spatial dynamics remains challenging. Existing methods often represent different scales within a shared latent space, potentially entangling smooth large-scale structures with localized fine-scale dynamics and making the latter difficult to capture. To address this limitation, we propose DESSNO, which explicitly separates latent fields into macroscopic and microscopic components using the discrete cosine transform (DCT) and evolves them through asymmetric spatial pathways. The macroscopic pathway captures broad spatial dependencies, while the microscopic pathway models localized structures using a dedicated transient evolution module. The two components are then recoupled through a content-adaptive spatial gate before global temporal propagation. This decompose-evolve-couple design turns explicit scale separation into structured dynamical routing, enabling scale-dependent evolution while preserving cross-scale interactions. Experiments on six challenging PDE benchmarks demonstrate that DESSNO consistently improves prediction accuracy and parameter efficiency, better preserves fine-scale structures, and remains stable over long-horizon autoregressive rollouts.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, Multiscale PDEs, Explicit Scale Separation, Discrete Cosine Transform, Spatiotemporal Attention

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KYkaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DESSNO for multiscale PDE prediction. It uses DCT to separate low- and high-frequency features, processes them through different attention pathways, and combines them using adaptive fusion. Experiments on six PDE benchmarks show improved prediction accuracy over the evaluated baselines. Ablation studies further examine the effectiveness of the proposed design.

Critical strengths
  1. The motivation for explicitly separating low- and high-frequency features is reasonable. The frequency-wise experiments also provide evidence that this design improves fine-scale prediction.

  2. The method achieves the best reported accuracy across six benchmarks, including comparisons with strong baselines such as DPOT and Poseidon. The additional ablation studies and longer rollout experiments further support its effectiveness.

Critical weaknesses

I believe the methodological novelty of this work needs further clarification. DCT-based frequency separation and scale-specific modeling have been explored in prior work. Therefore, I think the novelty should be more clearly demonstrated through the second contribution, particularly why these components are combined in this specific way.

Although the authors provide some ablation studies to demonstrate the effectiveness of individual components, they do not sufficiently explain or validate why these components need to interact in this particular way. For example, the necessity of Gated Fusion is not directly evaluated, and the placement of TEM exclusively in the Micro branch is not sufficiently justified through experiments. I believe these additional analyses would help clarify the unique contribution of the proposed method.

Questions for authors

Please refer to the Critical Weaknesses section.

Reviewer_nge3OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to alleviate the "representation burden" that neural PDE solvers encounter in multi-scale problems. To address a new architecture that, within each layer, first disentangles the large-scale and small-scale features, processes them independently, and couples them back. The experiments suggest that the proposed method demonstrates state-of-the-art performance across multiple benchmarking PDE datasets.

Critical strengths
  1. SOTA performance across a breadth of evaluation scenarios. DESSNO achieves the best performance across six benchmarks spanning shallow-water dynamics, compressible Euler flows, incompressible turbulence, real-world ERA5 wind forecasting, and three-dimensional MHD turbulence, compared with baselines including classical neural PDE solvers and foundation models.
Critical weaknesses
  1. The proposed method is motivated by problem-specific assumptions, but neither the assumptions nor the method design are sufficiently validated. (a) The motivation is that the representation is shared by features at different scales. For existing methods that use shared representation, e.g., U-Net-style and transformer-style methods, There's no direct evidence showing that interference between different scales exists and is harmful (Because the compared methods have many architectural differences, simply comparing the final score in some benchmarks is not sufficient). (b) For the proposed method, a critical ablation is missing that can assess the "harmful shared representation" hypothesis: add reasonable cross-scale interaction back into the DESSNO mode fairly (in terms of compute and parameter count), and see if the performance degrades.

  2. The contribution is limited. (a) Research problem. The background problem (multi-scale simulation) is generally relevant in scientific machine learning tasks. However, the paper is limited to a relatively standard problem setting, rather than actual challenges in multi-scale modeling, e.g., orders-of-magnitude scale separation. Evidence for that: experiments are mostly conventional PDE benchmarks. (b) Methodological contribution. The core idea of separating different scales has been explored by many methods, as is mentioned in the related work section. Thus, the main claim of the proposed method is that it finds a better network architecture. But the components of the methods are largely well known. Why the combination is effective, and how they interact with each other, remains obscure. The existing ablation studies have not established a solid methodological contribution.

Questions for authors
  1. What is the most important contribution or finding of this work from the author's perspective? Is it that separating representations of different scales is beneficial? If so, I believe this is an interesting finding, but stronger evidence is still required.

  2. Why is FNO classified into the shared representation category? In my understanding, the representation space of FNO is the Fourier basis space, where large-scale (low-frequency) and small-scale (high-frequency) information are independently stored.

  3. Are the baselines fairly tuned? Can you report the hyperparameter and checkpoint selection process?

(Below are questions of curiosity)

  1. Currently, the design is to separate the scale into two prescribed scales, and there is an assumption that low spatial scale corresponds to low temporal scale (high frequency). For systems that do not behave like this, can the current method be adapted?
Reviewer_aDKAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a neural operator architecture that essentially combines several existing ideas:

Decomposing the input features into low- and high-frequency components.

Using window-based attention to process local spatial regions in parallel, with different window sizes for the two frequency components.

Applying a temporal convolution to the high-frequency component. The motivation for this operation is unclear and potentially counterintuitive. Since temporal convolution mixes information from neighboring time steps, the authors should explain why this operation improves the prediction of high-frequency dynamics rather than smoothing out important transient information.

Combining the low- and high-frequency representations through a learned gating mechanism. However, Eq. (7) is not sufficiently explained. For example, the residual term (Z^{l-1}) is not clearly connected to the preceding definitions, making the computation difficult to follow.

Applying temporal attention after feature fusion. Again, Eq. (8) is presented without adequately defining the operators MSA and LN or explaining their precise implementation.

My main concerns are as follows:

Insufficient clarity and technical rigor. The methodological presentation contains unclear notation, insufficiently explained equations, and poorly motivated architectural choices. These issues make it difficult to understand and evaluate the proposed method rigorously.

Limited practical accuracy on several benchmarks. Although experiments are reported on six PDE benchmarks, only three achieve relative prediction errors below 5%. The remaining benchmarks exhibit considerably larger errors, raising questions about the practical accuracy and reliability of the proposed method. The authors should discuss whether these errors are acceptable for the corresponding physical applications.

Limited improvements in the more accurate prediction regimes. In Tables 2 and 3, the improvements on benchmarks where the errors are already relatively small are often marginal. This raises questions about the significance of the reported gains, particularly given the increased architectural complexity.

Unclear novelty and underlying mechanism. The proposed method appears to combine several established techniques, including frequency decomposition, local attention, temporal convolution, and adaptive feature fusion. Frequency bias in neural operators has been extensively studied, and multiple existing methods already address high-frequency learning through multiscale representations and frequency-aware architectures. The paper does not sufficiently distinguish its contributions from this literature or explain why the particular combination of components is necessary. The proposed design appears largely heuristic, with limited theoretical or experimental evidence establishing the underlying mechanism.

Overall Assessment

While the paper reports some encouraging numerical improvements, I find the novelty insufficiently established, the methodological presentation lacking in clarity, and the underlying design choices inadequately justified. Together with the limited practical accuracy on several benchmarks and the relatively modest improvements in the more accurate regimes, these concerns lead me to recommend rejection.

Critical strengths

None

Critical weaknesses
  1. Poor quality in writing with many undefined formulas. I can even understand the algorithm because of this.
  2. Very marginal improvement in numerical performance.
  3. No clear novelty.
Questions for authors
  1. How exactly is the algorithm implemented?
  2. What is the mathematic justification of the algorithm design?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.