- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SINO (Scale-Invariant Neural Operator) for learned closure modeling in coarse-grid PDE simulations. The method is motivated by two observations. First, although discretized physical fields are high-dimensional, turbulent dynamics may be governed by a relatively small set of dominant mechanisms. This suggests that the closure operator may admit a compact or low-rank representation. Second, the underlying physical laws should remain consistent across different grid resolutions. Based on these motivations, SINO adopts a dual-branch neural operator. The frequency branch captures global spectral interactions, while the spatial branch models local nonlinear structures. Instead of learning an independent kernel for each Fourier mode or spatial offset, both branches use bottleneck hypernetworks to generate continuous kernels conditioned on normalized coordinates. The authors argue that this design incorporates low-rank structure, scale invariance, and complementary global-local inductive biases. Across several closure-modeling benchmarks, SINO is compared with a range of existing baselines and achieves competitive empirical performance.
Critical strengths
-
The architecture is simple and well motivated. SINO also demonstrates a strong parameter–accuracy trade-off across the evaluated benchmarks.
-
The paper analyzes learned closure models from several complementary perspectives. In particular, the energy-spectrum experiment provides a more physics-oriented evaluation than trajectory MSE alone. It examines whether the learned solver preserves the expected spectral behavior of decaying Burgers turbulence, including the characteristic energy decay across spatial frequencies.
Critical weaknesses
Weaknesses
The proposed coordinate normalization does not appear to guarantee the claimed physical scale invariance.
For the frequency branch, the paper defines
The paper argues that the same physical frequency receives the same representation across different resolutions. However, this interpretation does not appear to hold when the physical domain length is fixed. The physical Fourier wavenumber is
Therefore, the same physical wavenumber corresponds to the same Fourier mode index . For a fixed , however, changes with the grid resolution . For example,
while
Thus, this normalization appears to encode frequency relative to the grid Nyquist frequency, instead of the absolute physical frequency. If the intended notion is normalized frequency, the current terminology and physical interpretation should be revised accordingly.
A similar issue arises in the spatial branch. The paper uses , where denotes the discrete spatial offset and is the kernel radius. However, the corresponding physical displacement is
When the kernel size is fixed, such as the kernel used in the 2D experiments with , the same corresponds to a smaller physical displacement as increases. Therefore, represents relative position within the discrete kernel, but does not directly represent a resolution-invariant physical displacement. Since scale invariance is a central motivation of SINO, the paper should define more precisely what notion of physical scale is preserved by these normalizations.
The empirical evaluation does not directly test zero-shot generalization across grid resolutions.
Scale invariance is one of the main claims of the paper. However, the experiments do not include the most direct evaluation of this property: training at one resolution and evaluating zero-shot at an unseen resolution.
Such an experiment would provide much stronger evidence that SINO learns resolution-transferable operators instead of achieving parameter efficiency within each tested discretization.
The interpretation of the Lipschitz-continuity result appears stronger than what the theorem establishes.
The paper proves that the ReLU hypernetworks and are Lipschitz continuous, and uses this result to argue that the model avoids memorizing discrete grid patterns and overfitting high-frequency noise. However, Lipschitz continuity alone does not support these conclusions.
A standard finite ReLU MLP with finite weights is already Lipschitz continuous. A typical upper bound on its Lipschitz constant is
This constant can still be very large. Consequently, a Lipschitz network may exhibit sharp variations and can still memorize finite training data. The theorem therefore establishes the existence of a finite Lipschitz constant for the generated continuous kernels, but it does not by itself imply robustness against memorization or high-frequency overfitting. The theoretical claims should be stated more carefully, or additional assumptions and quantitative bounds should be provided to support these interpretations.
The related-work discussion should better cover prior work on frequency-dependent bottleneck representations in scientific machine learning.
In particular, the paper does not discuss F-Adapter [1], which studies parameter-efficient representations for Fourier-based large operator models. Although its setting is different from closure modeling, it is closely related to several motivations used in SINO. F-Adapter provides theoretical and empirical analysis of spectral sparsity in PDE solutions, energy concentration across frequency bands, bottleneck capacity and frequency-dependent parameter allocation.
This connection is relevant because prior evidence from large operator models suggests that the spectral complexity of PDE solutions is strongly non-uniform. Predictive information can be concentrated in energy-dominant frequency bands. This observation motivates frequency-aware bottleneck representations for parameter-efficient scientific learning. A discussion of F-Adapter would help clarify which aspects of SINO are novel and how its low-rank and frequency-aware motivations differ from existing approaches.
[1] Hangwei Zhang, Chun Kang, Yan Wang, and Difan Zou. F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning. NeurIPS 2025.
Questions for authors
Regarding the operator-norm analysis, could the authors provide a more rigorous justification for why the operator norm of FNO should grow linearly with the number of retained Fourier modes ?
For a Fourier multiplier, a more standard bound appears to be Under this formulation, the operator norm does not necessarily increase linearly with the number of retained modes. If the intended comparison concerns parameter count or model capacity instead of the operator norm, the corresponding statement should be revised.
I would also appreciate further clarification of the SINO bound. In particular, why can both and be treated as constants independent of as increases? If either quantity depends implicitly on , the claimed scaling advantage may require additional assumptions.
Regarding the Navier–Stokes simulation protocol, several numerical values reported for the Navier–Stokes simulations appear internally inconsistent.
For the forced Navier–Stokes experiment, the paper specifies
Direct evaluation gives approximately$8.77\times10^{-4},$ while the paper reports approximately The paper later reports together with 1000 saved frames, while the total production duration is stated as
These values do not appear to be mutually consistent. A similar issue occurs for the decaying Navier–Stokes experiment. The paper reports $ \Delta t_{\mathrm{frame}} \approx 1.17\times10^{-2}$ and 1000 saved frames, but gives a total production duration of only
Could the authors provide the exact simulation protocol for both settings, including the solver timestep, the number of solver timesteps between saved frames, the number of saved frames and the total simulated physical duration. These details are important for assessing the claimed long-term extrapolation results and for ensuring reproducibility.
All in all, regarding the weaknesses and questions I raised, if they are not adequately addressed, I will lower my recommendation score.
4 comments
Thanks for the authors' response. However, regarding W2, the new experiments do not establish whether the proposed coordinate normalization contributes to cross-resolution generalization. A controlled ablation comparing the current grid-relative coordinates with physical coordinates based on the wavenumber and displacement would help isolate this contribution. Both variants should be evaluated under identical zero-shot transfer settings, with matched model capacity and physical support. This comparison is necessary to substantiate the benefit of the proposed coordinate representation.
Response to Reviewer uEh9 on Coordinate Normalization Ablation
Thank you for this insightful suggestion. Following your recommendation, we conducted a controlled ablation study comparing our grid-relative coordinate normalization with unnormalized physical coordinates (wavenumber k and displacement x).
Experimental Setup:
- Variants compared: SINO (grid-relative normalized coordinates) vs. SINO(unnorm) (physical coordinates k, x)
- Controlled factors: Identical model capacity, physical support, hyperparameters, and random seeds
- Evaluation protocol: Zero-shot transfer under identical settings as suggested
Key Findings:
We are grateful for this suggestion, as it significantly strengthens our paper. We were pleasantly surprised to discover that coordinate normalization is indeed critical for cross-resolution generalization, particularly pronounced in 2D cases:
- Forcing Burgers (1D, 32→64): SINO(unnorm) degrades by 10.8× in full trajectory MSE (8.78E-03 vs. 8.12E-04)
- Forcing Navier-Stokes (2D, 64→128): SINO(unnorm) completely diverges across all metrics (interpolation, extrapolation, full), while SINO maintains stable generalization
These results isolate the contribution of grid-relative coordinate normalization and substantiate its necessity for transferable physical learning across resolutions.
Updated Manuscript:
We have updated the revised manuscript with these ablation results:
- Table 4 now includes the SINO(unnorm) baseline
- Section 4.3 (Zero-Shot Super-Resolution Generalization) discusses the ablation findings
Table 4: Zero-shot super-resolution generalization
MSE when training at coarse resolution and testing on finer grids without retraining. Interp: interpolation region (seen time range), Extrap: extrapolation region (unseen future), Full: entire test trajectory. Lower errors indicate more transferable physical learning.
Forcing Burgers (32→64):
| Model | Interp | Extrap | Full |
|---|---|---|---|
| FNO | 3.68E-04 | 1.97E-03 | 1.48E-03 |
| AM-FNO | 2.74E-04 | 3.55E-03 | 2.56E-03 |
| UFNO | 9.94E-03 | 5.74E-02 | 4.32E-02 |
| SINO(unnorm) | 2.48E-03 | 1.15E-02 | 8.78E-03 |
| SINO | 2.07E-04 | 1.07E-03 | 8.12E-04 |
Forcing Navier-Stokes Re=4000 (64→128):
| Model | Interp | Extrap | Full |
|---|---|---|---|
| FNO | 9.81E-03 | 5.08E-01 | 3.58E-01 |
| AM-FNO | 7.90E-02 | 1.23E+00 | 8.83E-01 |
| UFNO | 6.87E-03 | 4.09E-01 | 2.88E-01 |
| SINO(unnorm) | diverged | diverged | diverged |
| SINO | 3.99E-03 | 3.51E-01 | 2.47E-01 |
We believe these controlled experiments directly address your concern and demonstrate the isolated benefit of the proposed coordinate representation. Thank you again for helping us strengthen this aspect of our work.
Thanks for the author's detailed reply. Most of my concerns have been solved. I will keep my positive rating.
Clarifications on Scale Generalization, Theoretical Bounds, and Reproducibility
We thank the reviewer for the feedback. We added cross-resolution experiments, refined the theory and Related Work, and corrected the Navier–Stokes protocol.
1. Coordinate normalization and cross-resolution generalization
The original interpretation was too strong. Revised Section 3.1 clarifies that inputs are dimensionless, grid-relative coordinates for continuous kernel generation. The frequency coordinate
ξ=4k/N−1
is relative to the grid Nyquist frequency, not the absolute wavenumber
κk=2πk/L;
thus, the same physical wavenumber can map to different ξ values as N changes. Likewise,
ζ=j/R
is the relative position within the discrete kernel support, whereas physical displacement
Δx=jL/N
varies with resolution. These coordinates do not imply strict physical scale invariance: SINO targets shared, scale-persistent structures while allowing resolution-dependent closure details.
Section 4.3 now includes "Zero-Shot Super-Resolution Generalization", evaluating coarse-grid models on unseen finer grids with frozen weights and no retraining. For Forcing Burgers (32→64), SINO's full-trajectory MSE is
8.12×10−4,
versus
1.48×10−3
for FNO (1.8× lower). For Forcing NS at Re = 4000 (64→128), SINO's interpolation/extrapolation MSEs are
3.99×10−3/3.51×10−1,
versus
9.81×10−3/5.08×10−1
for FNO (2.5×/1.4× lower); full-trajectory MSEs are
\quad \text{and} \quad 3.58\times10^{-1}.$$ SINO achieves the lowest full-trajectory MSE among baselines on both benchmarks, supporting transfer of shared cross-resolution structures and physical regularities without implying strict physical scale invariance. **2. Lipschitz continuity** Lipschitz continuity alone cannot prevent memorization or high-frequency overfitting. Revised **Section 3.4** and **Appendix A.4.2** clarify that the theorem gives finite, weight-dependent kernel-sensitivity bounds for a fixed network. Large constants do not guarantee resistance to overfitting. Kernel PCA indicates compact low-dimensional parameterization, not physically interpretable components. **3. Related work** Section 2 now discusses **F-Adapter** (Zhang et al., NeurIPS 2025), F-FNO, scale-consistent learning, and EddyFormer. F-Adapter analyzes spectral sparsity, energy concentration, and frequency-dependent bottlenecks for fine-tuning pretrained operators. In contrast, SINO learns closure operators from scratch, generating continuous kernels through coordinate-conditioned bottleneck hypernetworks in parallel spectral and spatial branches. **4. Operator norms and parameterization complexity** The previous claim of linear FNO operator-norm growth was unjustified. Revised **Section 3.4** and **Appendix A.4.4** use the standard bound: $$\|K_{\mathrm{FNO}}\|_{2\to 2} \leq \sup_k \|R_k\|_2,$$ which does not imply linear growth with $k_{\max}$. The SINO frequency-branch bound is: $$\sqrt{n_f} \cdot \max_j \|B_j\|_2 \cdot \sup_{\xi} \|z_{\omega}(\xi)\|_2$$ An analogous bound holds for the spatial branch. These factors may depend on bottleneck dimension; the bounds are finite for fixed networks, not unconditional $$O(\sqrt{n_f}) \quad \text{or} \quad O(\sqrt{n_s})$$ guarantees. Thus, for fixed MLP hidden widths, dominant parameter counts are $$O(k_{\max}D^2)$$ for FNO, $$O(K^2D^2)$$ for CNNs, and $$O((n_f+n_s)D^2)$$ for SINO. **5. Navier–Stokes protocol and reproducibility** Revised **Appendix A.3, "Time Stepping Configuration,"** specifies DNS timesteps, steps/frame, snapshot intervals/counts, training timesteps, and warmup/production durations. The 1001 snapshots include the initial post-warmup state (1000 intervals). - **Forced NS (Re = 1000 and 4000):** The 512×512 DNS uses $$\delta t_{\mathrm{DNS}}=8.7656045\times10^{-4}$$ with 8 Forward Euler steps/frame, yielding $$\Delta t_{\mathrm{snap}}=7.0124836\times10^{-3}$$ and $$T_{\mathrm{prod}}=7.0124836.$$ Warmup is $$39.9930705$$ with total simulation time $$T_{\mathrm{sim}}=47.0056.$$ The 64×64 training solver takes one Forward Euler step plus pressure projection/frame, with $$\Delta t_{\mathrm{train}}=7.0124836\times10^{-3}.$$ - **Decaying NS:** The 512×512 DNS uses $$\delta t_{\mathrm{DNS}}=1.4609341\times10^{-3}$$ with 8 Forward Euler steps/frame, yielding $$\Delta t_{\mathrm{snap}}=1.1687473\times10^{-2}$$ and $$T_{\mathrm{prod}}=11.6874727.$$ Warmup is $$4.4923723$$ with total simulation time $$T_{\mathrm{sim}}=16.1798.$$ The 64×64 training solver takes one Forward Euler step plus pressure projection/frame, with $$\Delta t_{\mathrm{train}}=1.1687473\times10^{-2}.$$ Warmup is excluded from production snapshots. Data-generation and SINO-training code for all six PDE benchmarks is available at https://anonymous.4open.science/r/SINO-662D. We thank the reviewer for improving the manuscript.