- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes self-similar cascade decoders to recover fine-scale details attenuated by neural networks. The method shares a learned coarse-to-fine operator across scales and introduces Fourier and wavelet implementations for PDE prediction and image super-resolution.
Critical strengths
The paper provides a coherent formulation of self-similarity across scales, connecting the proposed operator to Fourier, wavelet, and transposed-convolution representations.
Critical weaknesses
- The experiments do not isolate the benefit of the proposed dilation-based design. In Table 6(b), translational and dilational variants achieve nearly identical performance, with ten-step relative L2 errors of 0.167 and 0.166, respectively. The gains may therefore come mainly from learned spectral amplitude correction rather than the specific scale relation emphasized by the theory.
- There is a gap between the theoretical analysis and the practical implementation. The theory characterizes frequency-support expansion, whereas S3C retains backbone-predicted phases and interpolates spectral amplitudes. Expanding frequency support does not establish accurate recovery of missing details, and the practical method remains dependent on the backbone’s phase predictions.
- The image super-resolution experiments provide limited evidence of cross-scale generalization. S2WC is trained and evaluated at ×2 and ×4, while ×3 is obtained by downsampling the ×4 output. No results are reported beyond the training scales, and the improvements over LTE are small and inconsistent across datasets.
Questions for authors
See Weaknesses.
10 comments
Furthermore, we use this zero-shot super-resolution setting to compare with a fixed power law extrapolation and with a per-shell MLP (parameter-matched). Compared to S3C, all three methods perform similarly at the seen resolutions, which supports our claim that S3C helps learn the power law spectral decay in this setting. However, S3C outperforms both on our zero-shot super-resolution ablation.
| Steps | rel. S3C | rel. Per-shell filter | rel. Power law | rel. S3C | rel. Per-shell filter | rel. Power law | rel. HF$_{32}$ S3C | rel. HF$_{32}$ Per-shell filter | rel. HF$_{32}$ Power law |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 21.7 | 49.5 | 22.5 | 55.2 | 92.2 | 57.2 | 99.0 | 224.3 | 102.4 |
| 2 | 24.3 | diverges | 25.1 | 60.9 | diverges | 63.2 | 103.5 | diverges | 106.9 |
| 4 | 30.4 | diverges | 31.9 | 72.8 | diverges | 76.8 | 112.1 | diverges | 117.8 |
| 6 | 39.0 | diverges | 41.4 | 89.2 | diverges | 95.0 | 122.6 | diverges | 131.1 |
| 8 | 48.9 | diverges | 51.8 | 106.3 | diverges | 112.5 | 131.7 | diverges | 141.6 |
| 10 | 58.7 | diverges | 61.4 | 120.3 | diverges | 125.6 | 137.6 | diverges | 147.4 |
We parameterize the per-shell MLP by passing in the normalized target wavenumber, but find that this parameterization is substantially less stable at out-of-domain extrapolation compared to S3C.
We also add new Kuramoto-Sivashinsky (KS) equation rollout results and compare FNO with dilation and translation on zero-shot super-resolution. Since KS exhibits an exponential decay, we expect translation to outperform dilation outside of distribution. Indeed, for models trained at resolution 64, we find improved performance with the former at resolution 256 (full results in Appendix F):
| Eval. res. | Steps | rel. Dilational | rel. Translational | rel. Dilational | rel. Translational | rel. HF$_{32}$ Dilational | rel. HF$_{32}$ Translational |
|---|---|---|---|---|---|---|---|
| 128 | 1 | 0.32 | 0.28 | 0.33 | 0.30 | 13.7 | 11.9 |
| 2 | 0.47 | 0.40 | 0.49 | 0.45 | 19.9 | 16.7 | |
| 4 | 0.57 | 0.47 | 0.60 | 0.55 | 23.6 | 19.2 | |
| 6 | 0.59 | 0.49 | 0.62 | 0.58 | 24.0 | 19.5 | |
| 8 | 0.60 | 0.51 | 0.64 | 0.60 | 24.1 | 19.5 | |
| 10 | 0.63 | 0.54 | 0.67 | 0.63 | 24.1 | 19.5 | |
| 256 | 1 | 0.32 | 0.27 | 0.98 | 0.82 | 13.5 | 11.5 |
| 2 | 0.47 | 0.39 | 1.41 | 1.14 | 19.5 | 16.1 | |
| 4 | 0.56 | 0.45 | 1.66 | 1.30 | 23.0 | 18.4 | |
| 6 | 0.58 | 0.47 | 1.70 | 1.32 | 23.4 | 18.7 | |
| 8 | 0.59 | 0.49 | 1.70 | 1.33 | 23.4 | 18.7 | |
| 10 | 0.61 | 0.52 | 1.71 | 1.35 | 23.4 | 18.7 |
Furthermore, we add a 1d synthetic setting to empirically validate the theory. We compare by directly instantiate the Fourier-space operator following the notation in Lemma 1: which can be computed as a matrix-vector multiplication across scales. Since the target is self-similar, we parameterize with a shared rule , with for . We directly parameterize a shared set of parameters , where the spatial bandwidth is defined such that for . Since the output is linear with respect to the parameters , we directly solve for the parameters using least squares, bypassing the need for gradient-based solutions.
Thanks for your time and for providing constructive feedback! We address each of your points below and have updated the manuscript in OpenReview with red text highlighting all additions.
Thanks for raising this point! We would first like to clarify the theoretical distinction between dilational and translation spacing, which was not made very clear in our first draft. Formally, Theorem 1 is a statement about the support of Sϕ; the corollary about λ=γ>1 holds true for the choice of the bands Bk introduced in Section 2.2. We focused on such Fourier bands because they are well-suited to the study of power law decays, which Navier-Stokes and natural images have been classically known to exhibit. In such cases, our theory predicts that dilational spacing should outperform translational spacing (see below for updated ablations on this). However, in the case of exponential decay, for instance, fixed-width Fourier bands may be a more suitable choice for analysis. In these cases, such as for the Kuramoto-Sivashinsky PDE, we would expect translational (fixed-width) spacing to outperform dilational spacing. We ablate both of these in the results that follow.
Since this was unclear in the original draft, we have updated Section 2.2 and the informal statement of Theorem 1 to reflect this distinction. We have also added an Example 2 on learning an exponential decay accompanying the existing Example 1 for learning a power law decay. Please refer to the revised main text for details.
Next, we would also like to make a minor correction to our comparison between dilational and translational spacing. In the initial version of our Table 6, at training resolution dilational vs. translational performed approximately the same, within the variance threshold between runs. We have since found that this occurs because at the 64×64 training resolution, the band spacing of dilation and translation are nearly identical. As such, they perform very similar operations when evaluated at training resolution. However, at unseen higher resolutions, the band spacing of translation and dilation no longer coincide. As such, we ablate difference between dilation and translation spacing for Navier-Stokes on FNO trained at 64×64 and evaluated zero-shot at 128×128. Since Navier-Stokes exhibits a power law decay, a dyadic decomposition (Bk as defined in Section 2) is well-suited to model its spectrum. As such, we expect dilational spacing to outperform translational spacing on the unseen resolutions, which is indeed what we observe: