acceptodds
Under review as a conference paper at ICLR 2027

SINO: Scale-Invariant Neural Operator

Abstract

In scientific machine learning, physical fields governed by partial differential equations exhibit low-rank structure and scale invariance. When solving equations on coarse grids, missing information leads to the closure problem—modeling unresolved physics to recover lost dynamics. Although closure terms depend on grid resolution, they represent scale-invariant physical laws. A model truly learning physics should capture these mechanisms with low-rank parameterization rather than memorizing grid-specific patterns. Inspired by this, we propose the Scale-Invariant Neural Operator (SINO), which learns on normalized physical scales via a dual-branch architecture operating in spectral and spatial domains. SINO uses bottleneck MLPs to generate continuous convolution kernels, embedding explicit low-rank inductive bias that concentrates 95% variance in 2-3 modes, as validated by PCA across benchmarks, while drastically reducing parameters. This principled design yields 38 steeper scaling law exponents than FNO, demonstrating superior parameter efficiency. We compare SINO with traditional models (U-Net, DeepONet), Transformer models (Transolver, Oformer, GK-Transformer), and frequency-domain models (FNO, AMFNO, UFNO) on closure problems spanning externally forced Burgers turbulence, decaying Burgers turbulence, KS turbulence, Kolmogorov-forced NS turbulence, and decaying NS turbulence. Experiments show SINO achieves 1.5–38 error reduction and 2–23 parameter efficiency over baselines, with superior scaling laws reflecting exceptional data efficiency from principled low-rank design. Code is available at https://anonymous.4open.science/r/SINO-662D.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific Machine Learning, Neural Operator, Scale Invariance, Partial Differential Equations, Turbulence Modeling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uEh9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SINO (Scale-Invariant Neural Operator) for learned closure modeling in coarse-grid PDE simulations. The method is motivated by two observations. First, although discretized physical fields are high-dimensional, turbulent dynamics may be governed by a relatively small set of dominant mechanisms. This suggests that the closure operator may admit a compact or low-rank representation. Second, the underlying physical laws should remain consistent across different grid resolutions. Based on these motivations, SINO adopts a dual-branch neural operator. The frequency branch captures global spectral interactions, while the spatial branch models local nonlinear structures. Instead of learning an independent kernel for each Fourier mode or spatial offset, both branches use bottleneck hypernetworks to generate continuous kernels conditioned on normalized coordinates. The authors argue that this design incorporates low-rank structure, scale invariance, and complementary global-local inductive biases. Across several closure-modeling benchmarks, SINO is compared with a range of existing baselines and achieves competitive empirical performance.

Critical strengths
  1. The architecture is simple and well motivated. SINO also demonstrates a strong parameter–accuracy trade-off across the evaluated benchmarks.

  2. The paper analyzes learned closure models from several complementary perspectives. In particular, the energy-spectrum experiment provides a more physics-oriented evaluation than trajectory MSE alone. It examines whether the learned solver preserves the expected spectral behavior of decaying Burgers turbulence, including the characteristic energy decay across spatial frequencies.

Critical weaknesses

Weaknesses

The proposed coordinate normalization does not appear to guarantee the claimed physical scale invariance.

For the frequency branch, the paper defines

ξ=4kN−1.\xi=\frac{4k}{N}-1.

The paper argues that the same physical frequency receives the same representation across different resolutions. However, this interpretation does not appear to hold when the physical domain length LL is fixed. The physical Fourier wavenumber is

κk=2πkL.\kappa_k=\frac{2\pi k}{L}.

Therefore, the same physical wavenumber corresponds to the same Fourier mode index kk. For a fixed kk, however, ξ\xi changes with the grid resolution NN. For example,

k=4,N=64⇒ξ=−0.75,k=4, N=64 \Rightarrow \xi=-0.75,

while

k=4,N=128⇒ξ=−0.875.k=4, N=128 \Rightarrow \xi=-0.875.

Thus, this normalization appears to encode frequency relative to the grid Nyquist frequency, instead of the absolute physical frequency. If the intended notion is normalized frequency, the current terminology and physical interpretation should be revised accordingly.

A similar issue arises in the spatial branch. The paper uses ζ=jR\zeta=\frac{j}{R}, where jj denotes the discrete spatial offset and RR is the kernel radius. However, the corresponding physical displacement is Δx=jLN.\Delta x=\frac{jL}{N}.

When the kernel size is fixed, such as the 9×99\times9 kernel used in the 2D experiments with R=4R=4, the same jj corresponds to a smaller physical displacement as NN increases. Therefore, ζ\zeta represents relative position within the discrete kernel, but does not directly represent a resolution-invariant physical displacement. Since scale invariance is a central motivation of SINO, the paper should define more precisely what notion of physical scale is preserved by these normalizations.

The empirical evaluation does not directly test zero-shot generalization across grid resolutions.

Scale invariance is one of the main claims of the paper. However, the experiments do not include the most direct evaluation of this property: training at one resolution and evaluating zero-shot at an unseen resolution.

Such an experiment would provide much stronger evidence that SINO learns resolution-transferable operators instead of achieving parameter efficiency within each tested discretization.

The interpretation of the Lipschitz-continuity result appears stronger than what the theorem establishes.

The paper proves that the ReLU hypernetworks Φω\Phi_\omega and Φx\Phi_x are Lipschitz continuous, and uses this result to argue that the model avoids memorizing discrete grid patterns and overfitting high-frequency noise. However, Lipschitz continuity alone does not support these conclusions.

A standard finite ReLU MLP with finite weights is already Lipschitz continuous. A typical upper bound on its Lipschitz constant is

L≤∏ℓ∥Wℓ∥2.L \leq \prod_{\ell}\|W_\ell\|_2.

This constant can still be very large. Consequently, a Lipschitz network may exhibit sharp variations and can still memorize finite training data. The theorem therefore establishes the existence of a finite Lipschitz constant for the generated continuous kernels, but it does not by itself imply robustness against memorization or high-frequency overfitting. The theoretical claims should be stated more carefully, or additional assumptions and quantitative bounds should be provided to support these interpretations.

The related-work discussion should better cover prior work on frequency-dependent bottleneck representations in scientific machine learning.

In particular, the paper does not discuss F-Adapter [1], which studies parameter-efficient representations for Fourier-based large operator models. Although its setting is different from closure modeling, it is closely related to several motivations used in SINO. F-Adapter provides theoretical and empirical analysis of spectral sparsity in PDE solutions, energy concentration across frequency bands, bottleneck capacity and frequency-dependent parameter allocation.

This connection is relevant because prior evidence from large operator models suggests that the spectral complexity of PDE solutions is strongly non-uniform. Predictive information can be concentrated in energy-dominant frequency bands. This observation motivates frequency-aware bottleneck representations for parameter-efficient scientific learning. A discussion of F-Adapter would help clarify which aspects of SINO are novel and how its low-rank and frequency-aware motivations differ from existing approaches.

[1] Hangwei Zhang, Chun Kang, Yan Wang, and Difan Zou. F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning. NeurIPS 2025.

Questions for authors

Regarding the operator-norm analysis, could the authors provide a more rigorous justification for why the L2L_2 operator norm of FNO should grow linearly with the number of retained Fourier modes kmax⁡k_{\max}?

For a Fourier multiplier, a more standard L2L_2 bound appears to be ∥K∥2→2≤sup⁡k∥Rk∥2.\|\mathcal{K}\|_{2\to2} \leq \sup_k \|R_k\|_2. Under this formulation, the operator norm does not necessarily increase linearly with the number of retained modes. If the intended comparison concerns parameter count or model capacity instead of the operator norm, the corresponding statement should be revised.

I would also appreciate further clarification of the SINO bound. In particular, why can both max⁡j∥Bj∥2\max_j \lVert B_j \rVert_2 and sup⁡ξ∥zω(ξ)∥2\sup_{\xi} \lVert z_{\omega}(\xi) \rVert_2 be treated as constants independent of nfn_f as nfn_f increases? If either quantity depends implicitly on nfn_f, the claimed scaling advantage may require additional assumptions.

Regarding the Navier–Stokes simulation protocol, several numerical values reported for the Navier–Stokes simulations appear internally inconsistent.

For the forced Navier–Stokes experiment, the paper specifies Δt=0.52π/5127.\Delta t = 0.5\frac{2\pi/512}{7}.

Direct evaluation gives approximately$8.77\times10^{-4},$ while the paper reports approximately 5.59×10−4.5.59\times10^{-4}. The paper later reports Δtframe≈4.47×10−3\Delta t_{\mathrm{frame}} \approx 4.47\times10^{-3} together with 1000 saved frames, while the total production duration is stated as Tproduction=0.0448.T_{\mathrm{production}}=0.0448.

These values do not appear to be mutually consistent. A similar issue occurs for the decaying Navier–Stokes experiment. The paper reports $ \Delta t_{\mathrm{frame}} \approx 1.17\times10^{-2}$ and 1000 saved frames, but gives a total production duration of only Tproduction≈0.0117.T_{\mathrm{production}} \approx 0.0117.

Could the authors provide the exact simulation protocol for both settings, including the solver timestep, the number of solver timesteps between saved frames, the number of saved frames and the total simulated physical duration. These details are important for assessing the claimed long-term extrapolation results and for ensuring reproducibility.


All in all, regarding the weaknesses and questions I raised, if they are not adequately addressed, I will lower my recommendation score.

4 comments
AuthorsOpenReview ↗

Clarifications on Scale Generalization, Theoretical Bounds, and Reproducibility

We thank the reviewer for the feedback. We added cross-resolution experiments, refined the theory and Related Work, and corrected the Navier–Stokes protocol.

1. Coordinate normalization and cross-resolution generalization

The original interpretation was too strong. Revised Section 3.1 clarifies that inputs are dimensionless, grid-relative coordinates for continuous kernel generation. The frequency coordinate

ξ=4k/N−1\xi=4k/N-1

is relative to the grid Nyquist frequency, not the absolute wavenumber

κk=2πk/L;\kappa_k=2\pi k/L;

thus, the same physical wavenumber can map to different ξ\xi values as NN changes. Likewise,

ζ=j/R\zeta=j/R

is the relative position within the discrete kernel support, whereas physical displacement

Δx=jL/N\Delta x=jL/N

varies with resolution. These coordinates do not imply strict physical scale invariance: SINO targets shared, scale-persistent structures while allowing resolution-dependent closure details.

Section 4.3 now includes "Zero-Shot Super-Resolution Generalization", evaluating coarse-grid models on unseen finer grids with frozen weights and no retraining. For Forcing Burgers (32→64), SINO's full-trajectory MSE is

8.12×10−4,8.12\times10^{-4},

versus

1.48×10−31.48\times10^{-3}

for FNO (1.8× lower). For Forcing NS at Re = 4000 (64→128), SINO's interpolation/extrapolation MSEs are

3.99×10−3/3.51×10−1,3.99\times10^{-3}/3.51\times10^{-1},

versus

9.81×10−3/5.08×10−19.81\times10^{-3}/5.08\times10^{-1}

for FNO (2.5×/1.4× lower); full-trajectory MSEs are

\quad \text{and} \quad 3.58\times10^{-1}.$$ SINO achieves the lowest full-trajectory MSE among baselines on both benchmarks, supporting transfer of shared cross-resolution structures and physical regularities without implying strict physical scale invariance. **2. Lipschitz continuity** Lipschitz continuity alone cannot prevent memorization or high-frequency overfitting. Revised **Section 3.4** and **Appendix A.4.2** clarify that the theorem gives finite, weight-dependent kernel-sensitivity bounds for a fixed network. Large constants do not guarantee resistance to overfitting. Kernel PCA indicates compact low-dimensional parameterization, not physically interpretable components. **3. Related work** Section 2 now discusses **F-Adapter** (Zhang et al., NeurIPS 2025), F-FNO, scale-consistent learning, and EddyFormer. F-Adapter analyzes spectral sparsity, energy concentration, and frequency-dependent bottlenecks for fine-tuning pretrained operators. In contrast, SINO learns closure operators from scratch, generating continuous kernels through coordinate-conditioned bottleneck hypernetworks in parallel spectral and spatial branches. **4. Operator norms and parameterization complexity** The previous claim of linear FNO operator-norm growth was unjustified. Revised **Section 3.4** and **Appendix A.4.4** use the standard bound: $$\|K_{\mathrm{FNO}}\|_{2\to 2} \leq \sup_k \|R_k\|_2,$$ which does not imply linear growth with $k_{\max}$. The SINO frequency-branch bound is: $$\sqrt{n_f} \cdot \max_j \|B_j\|_2 \cdot \sup_{\xi} \|z_{\omega}(\xi)\|_2$$ An analogous bound holds for the spatial branch. These factors may depend on bottleneck dimension; the bounds are finite for fixed networks, not unconditional $$O(\sqrt{n_f}) \quad \text{or} \quad O(\sqrt{n_s})$$ guarantees. Thus, for fixed MLP hidden widths, dominant parameter counts are $$O(k_{\max}D^2)$$ for FNO, $$O(K^2D^2)$$ for CNNs, and $$O((n_f+n_s)D^2)$$ for SINO. **5. Navier–Stokes protocol and reproducibility** Revised **Appendix A.3, "Time Stepping Configuration,"** specifies DNS timesteps, steps/frame, snapshot intervals/counts, training timesteps, and warmup/production durations. The 1001 snapshots include the initial post-warmup state (1000 intervals). - **Forced NS (Re = 1000 and 4000):** The 512×512 DNS uses $$\delta t_{\mathrm{DNS}}=8.7656045\times10^{-4}$$ with 8 Forward Euler steps/frame, yielding $$\Delta t_{\mathrm{snap}}=7.0124836\times10^{-3}$$ and $$T_{\mathrm{prod}}=7.0124836.$$ Warmup is $$39.9930705$$ with total simulation time $$T_{\mathrm{sim}}=47.0056.$$ The 64×64 training solver takes one Forward Euler step plus pressure projection/frame, with $$\Delta t_{\mathrm{train}}=7.0124836\times10^{-3}.$$ - **Decaying NS:** The 512×512 DNS uses $$\delta t_{\mathrm{DNS}}=1.4609341\times10^{-3}$$ with 8 Forward Euler steps/frame, yielding $$\Delta t_{\mathrm{snap}}=1.1687473\times10^{-2}$$ and $$T_{\mathrm{prod}}=11.6874727.$$ Warmup is $$4.4923723$$ with total simulation time $$T_{\mathrm{sim}}=16.1798.$$ The 64×64 training solver takes one Forward Euler step plus pressure projection/frame, with $$\Delta t_{\mathrm{train}}=1.1687473\times10^{-2}.$$ Warmup is excluded from production snapshots. Data-generation and SINO-training code for all six PDE benchmarks is available at https://anonymous.4open.science/r/SINO-662D. We thank the reviewer for improving the manuscript.

Thanks for the authors' response. However, regarding W2, the new experiments do not establish whether the proposed coordinate normalization contributes to cross-resolution generalization. A controlled ablation comparing the current grid-relative coordinates (ξ,ζ)(\xi,\zeta) with physical coordinates based on the wavenumber κk\kappa_k and displacement Δx\Delta x would help isolate this contribution. Both variants should be evaluated under identical zero-shot transfer settings, with matched model capacity and physical support. This comparison is necessary to substantiate the benefit of the proposed coordinate representation.

AuthorsOpenReview ↗

Response to Reviewer uEh9 on Coordinate Normalization Ablation

Thank you for this insightful suggestion. Following your recommendation, we conducted a controlled ablation study comparing our grid-relative coordinate normalization with unnormalized physical coordinates (wavenumber k and displacement x).

Experimental Setup:

  • Variants compared: SINO (grid-relative normalized coordinates) vs. SINO(unnorm) (physical coordinates k, x)
  • Controlled factors: Identical model capacity, physical support, hyperparameters, and random seeds
  • Evaluation protocol: Zero-shot transfer under identical settings as suggested

Key Findings:

We are grateful for this suggestion, as it significantly strengthens our paper. We were pleasantly surprised to discover that coordinate normalization is indeed critical for cross-resolution generalization, particularly pronounced in 2D cases:

  • Forcing Burgers (1D, 32→64): SINO(unnorm) degrades by 10.8× in full trajectory MSE (8.78E-03 vs. 8.12E-04)
  • Forcing Navier-Stokes (2D, 64→128): SINO(unnorm) completely diverges across all metrics (interpolation, extrapolation, full), while SINO maintains stable generalization

These results isolate the contribution of grid-relative coordinate normalization and substantiate its necessity for transferable physical learning across resolutions.

Updated Manuscript:

We have updated the revised manuscript with these ablation results:

  • Table 4 now includes the SINO(unnorm) baseline
  • Section 4.3 (Zero-Shot Super-Resolution Generalization) discusses the ablation findings

Table 4: Zero-shot super-resolution generalization

MSE when training at coarse resolution and testing on finer grids without retraining. Interp: interpolation region (seen time range), Extrap: extrapolation region (unseen future), Full: entire test trajectory. Lower errors indicate more transferable physical learning.

Forcing Burgers (32→64):

ModelInterpExtrapFull
FNO3.68E-041.97E-031.48E-03
AM-FNO2.74E-043.55E-032.56E-03
UFNO9.94E-035.74E-024.32E-02
SINO(unnorm)2.48E-031.15E-028.78E-03
SINO2.07E-041.07E-038.12E-04

Forcing Navier-Stokes Re=4000 (64→128):

ModelInterpExtrapFull
FNO9.81E-035.08E-013.58E-01
AM-FNO7.90E-021.23E+008.83E-01
UFNO6.87E-034.09E-012.88E-01
SINO(unnorm)divergeddivergeddiverged
SINO3.99E-033.51E-012.47E-01

We believe these controlled experiments directly address your concern and demonstrate the isolated benefit of the proposed coordinate representation. Thank you again for helping us strengthen this aspect of our work.

Thanks for the author's detailed reply. Most of my concerns have been solved. I will keep my positive rating.

Reviewer_128aOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces SINO, a hybrid FNO-CNN architecture for turbulence closure learning. The core idea is to learn a low-rank representation from a bottlenecked meta network and feed it into the spectral branch. The authors argue this design gives low-rank and scale-invariant inductive biases. The authors provide Lipschitz and operator-norm bounds on the low-rank structure, and conducted PCA to show that low-rank structure exists in trained networks. Experiment results show accuracy benefit on 2D turbulence benchmarks.

Critical strengths
  1. The network is well-motivated by scale invariance and scale separation principles of turbulence.
  2. The authors provided a carefully-engineered network architecture that achieves better accuracy and scaling.
Critical weaknesses
  1. The motivations are not well reflected in the actual method. The main motivation is to learn a closure model to assist the network, but there's no scale separation throughout the design. The spectral branch and the local branch operate on the same scale, which means that the low-rank features that the local branch learned does not necessarily corresponds to "unresolved sub grid-scale dynamics". I acknowledge that spectral branch is global, so this type of localized feature extraction inside FNO can be complementary to the architecture. Given that scale-separated network architecture has been explored in ML community, so the authors need to explicitly ablate if the low-rank limitation helps the network in either scaling or generalization better.
  2. Scale invariance is only empirically shown in Fig. 6. The authors show that SINO has a smooth behavior when the grid is cut off inside the inertial range. However, both DNS and Baseline's spectra look like a cut off instead of aliasing. I do not know how interpret this as scale invariance.
Questions for authors
  1. Can authors clarify what is the role of closure learning in their architecture?
  2. Can authors discuss the SINO's relation to low-rank models (https://arxiv.org/abs/2111.13802), multiscale models (e.g., https://arxiv.org/abs/2510.24173) and ablate the role of low-rank?
  3. Can authors provide evidence (e.g., visually) of SINO's benefit in the grid cut-off experiment?
2 comments
AuthorsOpenReview ↗

Clarifications on Closure Learning, Low-Rank Design, and Spectral Fidelity

We thank the reviewer for the constructive comments. We expanded Related Work, added a bottleneck-width ablation, and clarified the physical interpretation of our experiments.

Q1. Role of closure learning in SINO

Following the Indirect Neural Corrector (INC) formulation (Wei et al., NeurIPS 2025; https://doi.org/10.52202/085713-3681), SINO learns a closure correction within a coarse-grid solver. At time step n, SINO predicts a correction from the current coarse-grid state:

τΔn=SINO(uˉΔn;Θ)\tau_\Delta^n = \mathrm{SINO}(\bar{u}_\Delta^n; \Theta)

The coarse-grid dynamics then become:

duˉΔdt=LΔ(uˉΔ)+τΔn\frac{d\bar{u}_\Delta}{dt} = \mathcal{L}_\Delta(\bar{u}_\Delta) + \tau_\Delta^n

where ℒ_Δ represents the coarse-grid physical operator. Unlike direct state correction, SINO adds its prediction to the right-hand side (RHS), which is integrated by the numerical solver. Thus, the correction represents the net effect of unresolved physics on resolved coarse-grid dynamics, rather than an explicitly reconstructed subgrid velocity field.

SINO uses autoregressive solver-in-the-loop training with progressively increasing rollout horizons. For multi-stage integrators, the correction is reused across internal stages of each time step. The frequency and spatial branches provide complementary global and local representations; neither is claimed to implement explicit physical scale separation.

Q2. Relation to low-rank and multiscale operators; bottleneck ablation

We expanded Section 2 (Related Work) to discuss F-FNO (Tran et al., 2023) and EddyFormer (Du & Krishnapriyan, 2025). F-FNO factorizes Fourier operations across spatial dimensions, whereas SINO uses coordinate-conditioned bottleneck hypernetworks to constrain generated kernels to low-dimensional basis subspaces. EddyFormer explicitly decomposes grid-scale and subgrid-scale components through spectral-element tokenization. In contrast, SINO learns an RHS closure from coarse-grid states using parallel spectral and spatial branches, without explicitly decomposing physical scales.

We added Appendix A.5.5, "Ablation Study: Low-Rank Bottleneck Efficiency," varying the hidden dimension from d = 2 to 128 on three benchmarks while keeping other components fixed.

Notably, d = 2 already achieves competitive MSEs of 1.03E-03, 1.33E-03, and 4.37E-03 on Forcing Burgers, KS, and Decaying Burgers, respectively, with only 6,681 parameters. Increasing d to 128 requires 30.6 times more parameters (204,753), reducing errors by 29.8%, 61.4%, and 23.6%, respectively. These gains are disproportionate to the increase in model capacity. In particular, increasing d from 64 to 128 improves KS by only 0.4%, while Decaying Burgers deteriorates from 3.15E-03 to 3.34E-03.

These results demonstrate that even a very narrow bottleneck (d = 2) provides an effective, parameter-efficient kernel representation. Wider bottlenecks offer diminishing and problem-dependent returns, supporting our compact low-rank design.

Q3. Evidence in the grid-cutoff experiment

We agree that Figure 6 shows spectral truncation rather than aliasing, and do not interpret it as evidence of aliasing suppression. The experiment compares predicted spectra against high-resolution DNS (N = 2048) around and beyond the coarse-grid Nyquist cutoff.

Conventional coarse-grid simulations exhibit numerical dissipation, while neural baselines show excessive high-frequency attenuation. SINO produces spectra closer to DNS and better preserves the k⁻² inertial-range scaling before the physical dissipation range. Its advantage therefore concerns spectral reconstruction and high-wavenumber dissipation, not eliminating spectral cutoffs.

Quantitative evidence is provided in Appendix A.5.6, "Spectral Fidelity Analysis" (Table 10). SINO achieves high-frequency log-spectrum errors of 0.015, 0.023, 0.079, and 0.180 at resampling factors 2, 4, 8, and 16, respectively, compared with 0.122, 0.204, 0.294, and 0.561 for baselines, corresponding to 3.1–8.9 times lower errors.

Additionally, Appendix A.5.4, Figure 12 provides qualitative evidence on Decaying Burgers turbulence. Its left and right columns show spatiotemporal predictions and corresponding error distributions, respectively. SINO exhibits smaller errors and smoother error patterns than the baselines.

We thank the reviewer again for helping us clarify SINO's closure learning formulation, low-rank design, and spectral fidelity.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer 128a,

Thank you again for your time and constructive feedback. We have carefully addressed the concerns raised in your review through detailed explanations, additional experiments, and corresponding manuscript revisions. We believe that our responses have addressed most of the concerns you raised.

As the rebuttal period is nearing its end, we would like to kindly follow up, as we have not yet received any further feedback from you. If you have any remaining comments or feedback regarding our responses, we would greatly appreciate hearing from you before the discussion period closes.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_UZYEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SINO, a neural closure model that augments coarse-grid PDE solvers with learned right-hand-side corrections. Its spectral and spatial branches use bottleneck MLPs to generate coordinate-dependent convolution kernels. Evaluations on six turbulence benchmarks show competitive rollout accuracy with compact models, supported by branch ablations and spectral comparisons.

Critical strengths
  1. The paper addresses a practically relevant problem: improving coarse-grid simulations without resolving all fine-scale dynamics. SINO achieves the lowest average error on five of the six main benchmarks, with substantially fewer parameters than the evaluated FNO and UFNO configurations.
  2. The combination of spectral and local spatial processing is well motivated. The branch ablations support their complementary roles, while the Burgers energy-spectrum results provide evidence of improved scale-dependent behavior beyond aggregate prediction error.
Critical weaknesses
  1. The evidence for scale invariance is less convincing than the fixed-resolution accuracy results. The cross-resolution experiments do not clearly establish whether a single trained model is transferred without retraining. Moreover, normalizing frequencies by grid resolution does not, by itself, preserve the representation of a fixed physical frequency. The low-dimensional kernel PCA results demonstrate compact parameterization, but do not establish that the learned components represent dominant physical mechanisms.
  2. The evaluation protocol leaves uncertainty about generalization and temporal robustness. Appendix A.3 states that the best model is selected using the test-window MSE, potentially introducing selection bias. The inconsistent timestep and total-duration descriptions also make it difficult to interpret the physical length of the reported rollouts and assess the strength of the long-term stability claims.
Questions for authors
  1. Do the cross-resolution results use the same frozen model weights? Please specify the training and testing resolutions and any retraining or adaptation. What precise invariance is intended by the normalized coordinates?
  2. Does test-window MSE determine checkpoint or hyperparameter selection? If so, are results available on a separate temporal window or independent trajectories not used for selection?
  3. What physical duration does the 1,000-step rollout represent, and how does it compare with a characteristic flow timescale? Please reconcile the CFL-based timestep, eight internal steps per saved frame, and reported production durations, and clarify the corresponding coarse-solver timestep.
1 comment
AuthorsOpenReview ↗

Clarifications on cross-resolution generalization, evaluation protocol, and time stepping

We thank the reviewer for the constructive comments. We revised the manuscript and added experiments to address these concerns.

Q1. Cross-resolution generalization and scale invariance

We clarify that the original cross-resolution experiments trained separate models at each resolution; no frozen model was transferred. Unlike a full PDE solution operator, a closure compensates for information unresolved at a given coarse-graining scale. At higher resolutions, more structures become resolved and the closure problem changes. Thus, a coarse-grid model cannot recover all fine-scale information on a finer grid. Nevertheless, different resolutions may share common physical structures.

To test transferability, we added Zero-Shot Super-Resolution Generalization in Section 4.3 of the revised manuscript. Coarse-grid models are evaluated on finer grids with frozen weights and no retraining or fine-tuning. SINO achieves the lowest full-trajectory MSE on both benchmarks: 8.12E-04 for Forcing Burgers (32 to 64), versus 1.48E-03 for FNO; and 2.47E-01 for Forcing NS (Re = 4000, 64 to 128), versus 3.58E-01 for FNO. On NS, SINO's interpolation and extrapolation errors are also 2.5 and 1.4 times lower, respectively. Full results, including AM-FNO and UFNO, are reported in the revised manuscript.

We clarify that ξ=4k/N−1\xi = 4k/N - 1 represents frequency relative to the grid Nyquist frequency, while ζ=j/R\zeta = j/R represents relative position within the discrete kernel support. As the reviewer notes, these coordinates do not preserve absolute physical wavenumbers or displacements across resolutions. Instead, they provide a continuous, grid-relative parameterization decoupled from the number of discretized modes and locations. Thus, scale invariance refers to this architectural inductive bias and the potential to capture shared structures, not strict physical-coordinate invariance or identical closure terms.

We agree that kernel PCA does not establish that learned components represent dominant physical mechanisms. We revised Section 3.4 and the corresponding appendix discussion to clarify that PCA indicates a compact low-dimensional parameterization of the learned kernels, without assigning physical meanings to individual components.

Q2. Checkpoint selection and unseen-trajectory generalization

The MSE on the extrapolation portion of the original test window is used for checkpoint selection within each fixed configuration, but not for hyperparameter tuning. All methods use the same protocol. We acknowledge its potential selection bias.

To independently evaluate generalization, we added Generalization on Unseen Trajectories in Appendix A.5.7 of the revised manuscript. We evaluate 64 entirely unseen trajectories for each of Forcing Burgers and Forcing Navier-Stokes (Re = 4000), excluded from both training and checkpoint selection. All 1000 saved snapshots per trajectory are used to calculate trajectory-averaged MSE. SINO achieves the lowest MSE on both benchmarks: 7.61E-04 on Burgers and 1.81E-01 on NS, versus 1.25E-03 and 2.60E-01 for AM-FNO, corresponding to reductions of 39.1% and 30.4%. Results for all baselines are included in the revised manuscript.

Q3. Physical duration, CFL-based time stepping, and rollout horizon

We revised Appendix A.3, Training Methodology, under "Time Stepping Configuration" to clarify the DNS timestep, saved-frame interval, coarse-grid timestep, and physical duration.

For forced NS (Re = 1000 and 4000), the 512 x 512 DNS uses the CFL-based timestep ΔtDNS=8.7656×10−4\Delta t_{\mathrm{DNS}} = 8.7656 \times 10^{-4}. Eight Forward Euler steps separate consecutive saved frames, giving Δtsnap=8×ΔtDNS=7.0125×10−3\Delta t_{\mathrm{snap}} = 8 \times \Delta t_{\mathrm{DNS}} = 7.0125 \times 10^{-3} and Tprod=1000×Δtsnap=7.0125T_{\mathrm{prod}} = 1000 \times \Delta t_{\mathrm{snap}} = 7.0125. The 64 x 64 coarse-grid solver advances once per saved-frame interval, with Δttrain=Δtsnap=7.0125×10−3\Delta t_{\mathrm{train}} = \Delta t_{\mathrm{snap}} = 7.0125 \times 10^{-3}. The eight internal steps belong to DNS data generation, not the coarse-grid solver.

For forced flow, kf=4k_f = 4 and Umax⁡=7U_{\max} = 7, giving the nominal forcing-scale advective time Tf=2π/(kf×Umax⁡)≈0.2244T_f = 2\pi/(k_f \times U_{\max}) \approx 0.2244. The production rollout spans approximately 31.25 forcing-scale turnover times, or 7.81 domain-scale crossing times. For decaying NS, 1000 production intervals correspond to Tprod=11.6875T_{\mathrm{prod}} = 11.6875, approximately 7.81 nominal domain-scale crossing times based on Umax⁡=4.2U_{\max} = 4.2.

The warmup phase is excluded from the 1001 recorded production snapshots, which define 1000 production intervals; total simulation duration includes both warmup and production.

Finally, we released the data-generation and SINO training codes for all six PDE benchmarks: https://anonymous.4open.science/r/SINO-662D

We thank the reviewer for helping improve the manuscript's clarity, rigor, and reproducibility.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive feedback. Reviewer UZYE highlighted the relevance of coarse-grid closure modeling, SINO's accuracy with fewer parameters, and its complementary spectral and spatial branches. Reviewer 128a recognized the carefully engineered architecture, its accuracy and scaling, and its turbulence-inspired motivation. Reviewer uEh9 praised its simplicity, parameter–accuracy trade-off, and physics-oriented spectral analysis. In response, we conducted additional experiments and revised the manuscript. The revised manuscript has been updated with all modifications highlighted in blue, and our anonymous code repository has also been updated. The major changes are summarized below.

  1. Clarification of closure learning. We clarify that SINO predicts a solver-integrated right-hand-side correction representing the net effect of unresolved physics, rather than explicitly reconstructing subgrid velocity fields. Its frequency and spatial branches provide complementary global and local representations, not explicit physical scale separation. Please see our response to Reviewer 128a.
  2. Cross-resolution generalization. We added Zero-Shot Super-Resolution Generalization to Section 4.3, evaluating coarse-grid models on unseen finer grids without retraining. SINO achieves the lowest full-trajectory MSE on Forcing Burgers (32→64; 8.12E-04 vs. 1.48E-03 for FNO) and Forcing NS (Re=4000, 64→128; 2.47E-01 vs. 3.58E-01 for FNO). We revised Section 3.1 to clarify that ξ and ζ are dimensionless, grid-relative coordinates, not resolution-independent physical coordinates. SINO aims to capture shared, scale-persistent structures while allowing resolution-dependent closure details. Please see our responses to Reviewers UZYE and uEh9.
  3. Unseen-trajectory generalization. We clarify that extrapolation-window MSE is used for checkpoint selection, not hyperparameter tuning. We added experiments in Appendix A.5.7 evaluating 64 unseen trajectories per benchmark over 1,000 snapshots. SINO achieves the lowest MSE on Forcing Burgers (7.61E-04) and Forcing NS (Re=4000; 1.81E-01), reducing errors over AM-FNO by 39.1% and 30.4%. Please see our response to Reviewer UZYE.
  4. Bottleneck ablation and related work. We added a bottleneck-width ablation (d=2–128) in Appendix A.5.5 on three 1D benchmarks. Even d=2 achieves competitive performance with 6,681 parameters, while wider bottlenecks exhibit diminishing and problem-dependent returns. We expanded Section 2 to discuss F-FNO, F-Adapter, scale-consistent learning, and EddyFormer, clarifying SINO's distinctions from existing approaches. Please see our responses to Reviewers 128a and uEh9.
  5. Theoretical clarifications. We revised Section 3.4 and Appendix A.4.2 to clarify that Lipschitz continuity provides finite, weight-dependent sensitivity bounds but does not guarantee resistance to memorization or overfitting. Kernel PCA indicates compact parameterization, not physically interpretable components. We also revised Section 3.4 and Appendix A.4.4, correcting the FNO operator-norm comparison and clarifying the dependence of SINO's bounds on bottleneck dimensions. We compare parameterization complexity rather than claiming unconditional operator-norm scaling. Please see our responses to Reviewers UZYE and uEh9.
  6. Spectral fidelity. We clarify that the grid-cutoff experiment concerns spectral truncation and reconstruction, not aliasing suppression. Appendix A.5.6 (Table 10) reports 3.1–8.9× lower high-frequency log-spectrum errors for SINO. Appendix A.5.4 (Figure 12) provides additional Decaying Burgers predictions and error maps. Please see our response to Reviewer 128a.
  7. Navier–Stokes protocol and reproducibility. We revised Appendix A.3, Time Stepping Configuration, to clarify DNS and coarse-grid timesteps, snapshot intervals, warmup, and production durations. We also updated our anonymous repository with data-generation and SINO training codes for all six PDE benchmarks: https://anonymous.4open.science/r/SINO-662D. Please see our responses to Reviewers UZYE and uEh9.

We sincerely thank all reviewers for their valuable feedback, which has helped improve the clarity, rigor, and reproducibility of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.