acceptodds
Under review as a conference paper at ICLR 2027

One Rule for Every Scale: Recovering Fine Detail with Self-Similarity

Abstract

Neural networks modeling natural signals often correctly learn coarse spatial scales but struggle to resolve fine scales, leading to blur, missing detail, and rollout drift. This attenuation of fine detail has been observed independently across many domains, like image reconstruction and neural PDE solvers. We observe that many of these natural signals exhibit approximate self-similarity, meaning that their fine scales are largely predictable from their coarse scales. In this work, we propose to embed self-similarity as an architectural inductive bias to better resolve fine scales. We formalize this inductive bias as a recurrence with a shared operator across scales and prove when this operator can be parameterized efficiently with parameter count independent of the number of fine scales predicted, allowing it to be appended to existing models as a small module. Using this result, we introduce self-similar cascade decoders, a family of such modules that admit domain-specific realizations. Our Fourier basis instantiation, called the self-similar spectral cascade (S3C), improves performance on turbulent Navier-Stokes forecasting by up to 54% across backbones, resolutions, and time horizons—while contributing 0.1% of total parameters. Our wavelet instantiation, S2WC, outperforms several strong baselines for deterministic image super-resolution.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
self-similarity, spectral bias, super-resolution, operator learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_VNB4OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes self-similar cascade decoders to recover fine-scale details attenuated by neural networks. The method shares a learned coarse-to-fine operator across scales and introduces Fourier and wavelet implementations for PDE prediction and image super-resolution.

Critical strengths

The paper provides a coherent formulation of self-similarity across scales, connecting the proposed operator to Fourier, wavelet, and transposed-convolution representations.

Critical weaknesses
  1. The experiments do not isolate the benefit of the proposed dilation-based design. In Table 6(b), translational and dilational variants achieve nearly identical performance, with ten-step relative L2 errors of 0.167 and 0.166, respectively. The gains may therefore come mainly from learned spectral amplitude correction rather than the specific scale relation emphasized by the theory.
  2. There is a gap between the theoretical analysis and the practical implementation. The theory characterizes frequency-support expansion, whereas S3C retains backbone-predicted phases and interpolates spectral amplitudes. Expanding frequency support does not establish accurate recovery of missing details, and the practical method remains dependent on the backbone’s phase predictions.
  3. The image super-resolution experiments provide limited evidence of cross-scale generalization. S2WC is trained and evaluated at ×2 and ×4, while ×3 is obtained by downsampling the ×4 output. No results are reported beyond the training scales, and the improvements over LTE are small and inconsistent across datasets.
Questions for authors

See Weaknesses.

10 comments

Thanks for your time and for providing constructive feedback! We address each of your points below and have updated the manuscript in OpenReview with red text highlighting all additions.

The experiments do not isolate the benefit of the proposed dilation-based design ... The gains may therefore come mainly from learned spectral amplitude correction rather than the specific scale relation emphasized by the theory.

Thanks for raising this point! We would first like to clarify the theoretical distinction between dilational and translation spacing, which was not made very clear in our first draft. Formally, Theorem 1 is a statement about the support of Sϕ^\widehat{S\phi}; the corollary about λ=γ>1\lambda = \gamma > 1 holds true for the choice of the bands BkB_k introduced in Section 2.2. We focused on such Fourier bands because they are well-suited to the study of power law decays, which Navier-Stokes and natural images have been classically known to exhibit. In such cases, our theory predicts that dilational spacing should outperform translational spacing (see below for updated ablations on this). However, in the case of exponential decay, for instance, fixed-width Fourier bands may be a more suitable choice for analysis. In these cases, such as for the Kuramoto-Sivashinsky PDE, we would expect translational (fixed-width) spacing to outperform dilational spacing. We ablate both of these in the results that follow.

Since this was unclear in the original draft, we have updated Section 2.2 and the informal statement of Theorem 1 to reflect this distinction. We have also added an Example 2 on learning an exponential decay accompanying the existing Example 1 for learning a power law decay. Please refer to the revised main text for details.

Next, we would also like to make a minor correction to our comparison between dilational and translational spacing. In the initial version of our Table 6, at training resolution dilational vs. translational performed approximately the same, within the variance threshold between runs. We have since found that this occurs because at the 64×6464 \times 64 training resolution, the band spacing of dilation and translation are nearly identical. As such, they perform very similar operations when evaluated at training resolution. However, at unseen higher resolutions, the band spacing of translation and dilation no longer coincide. As such, we ablate difference between dilation and translation spacing for Navier-Stokes on FNO trained at 64×6464 \times 64 and evaluated zero-shot at 128×128128 \times 128. Since Navier-Stokes exhibits a power law decay, a dyadic decomposition (BkB_k as defined in Section 2) is well-suited to model its spectrum. As such, we expect dilational spacing to outperform translational spacing on the unseen resolutions, which is indeed what we observe:

Stepsrel. L2L^2 Dilationalrel. L2L^2 Translationalrel. H1H^1 Dilationalrel. H1H^1 Translationalrel. HF$_{32}$ Dilationalrel. HF$_{32}$ Translational
121.722.455.257.099.0102.3
224.324.960.963.0103.5107.1
430.431.172.875.2112.1116.2
639.040.089.292.1122.6127.7
848.950.2106.3109.6131.7138.0
1058.760.1120.4124.0137.7144.7

Furthermore, we use this zero-shot super-resolution setting to compare with a fixed power law extrapolation and with a per-shell MLP (parameter-matched). Compared to S3C, all three methods perform similarly at the seen resolutions, which supports our claim that S3C helps learn the power law spectral decay in this setting. However, S3C outperforms both on our zero-shot super-resolution ablation.

Stepsrel. L2L^2 S3Crel. L2L^2 Per-shell filterrel. L2L^2 Power lawrel. H1H^1 S3Crel. H1H^1 Per-shell filterrel. H1H^1 Power lawrel. HF$_{32}$ S3Crel. HF$_{32}$ Per-shell filterrel. HF$_{32}$ Power law
121.749.522.555.292.257.299.0224.3102.4
224.3diverges25.160.9diverges63.2103.5diverges106.9
430.4diverges31.972.8diverges76.8112.1diverges117.8
639.0diverges41.489.2diverges95.0122.6diverges131.1
848.9diverges51.8106.3diverges112.5131.7diverges141.6
1058.7diverges61.4120.3diverges125.6137.6diverges147.4

We parameterize the per-shell MLP by passing in the normalized target wavenumber, but find that this parameterization is substantially less stable at out-of-domain extrapolation compared to S3C.

We also add new Kuramoto-Sivashinsky (KS) equation rollout results and compare FNO with dilation and translation on zero-shot super-resolution. Since KS exhibits an exponential decay, we expect translation to outperform dilation outside of distribution. Indeed, for models trained at resolution 64, we find improved performance with the former at resolution 256 (full results in Appendix F):

Eval. res.Stepsrel. L2L^2 Dilationalrel. L2L^2 Translationalrel. H1H^1 Dilationalrel. H1H^1 Translationalrel. HF$_{32}$ Dilationalrel. HF$_{32}$ Translational
12810.320.280.330.3013.711.9
20.470.400.490.4519.916.7
40.570.470.600.5523.619.2
60.590.490.620.5824.019.5
80.600.510.640.6024.119.5
100.630.540.670.6324.119.5
25610.320.270.980.8213.511.5
20.470.391.411.1419.516.1
40.560.451.661.3023.018.4
60.580.471.701.3223.418.7
80.590.491.701.3323.418.7
100.610.521.711.3523.418.7

Furthermore, we add a 1d synthetic setting to empirically validate the theory. We compare λ∈{1,2}\lambda \in \{1, 2\} by directly instantiate the Fourier-space operator following the notation in Lemma 1: Sϕ^(ξ)=∑ζα~(ξ−λζ,ζ)ϕ^(ζ),\widehat{S\phi}(\xi) = \sum_{\zeta} \tilde \alpha(\xi - \lambda \zeta, \zeta) \hat{\phi}(\zeta), which can be computed as a matrix-vector multiplication across scales. Since the target is self-similar, we parameterize α~\tilde \alpha with a shared rule α~(η,ζ)=h(η,u(ζ))\tilde \alpha(\eta, \zeta) = h(\eta, u(\zeta)), with u(ζ)=ζ/2k∈B0u(\zeta) = \zeta / 2^k \in B_0 for ζ∈Bk\zeta \in B_k. We directly parameterize a shared set of parameters h∈C8×(2R+1)h \in \mathbb{C}^{8 \times (2R+1)}, where the spatial bandwidth RR is defined such that α~(η,ζ)=0\tilde \alpha(\eta, \zeta) = 0 for ∣η∣>R|\eta| > R. Since the output is linear with respect to the parameters hh, we directly solve for the parameters using least squares, bypassing the need for gradient-based solutions.

Reviewer_urTdOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+2)
Read review
Paper summary

The authors observe that for many natural signals—the paper specifically discusses PDE trajectories, PDE super-resolution, and image super-resolution—the energy from low to high frequencies exhibits self-similar scales from a frequency-domain perspective. Existing naive models, however, usually struggle to fit the high-frequency parts of these signals, i.e., spectral attenuation. Therefore, the authors first prove that, through coordinate dilation, the model can achieve this self-similar low-to-high-frequency energy extrapolation with the same fixed small set of parameters. They then propose plug-and-play model modules for two tasks, PDE simulation and image super-resolution, to implement this operation in practice. Inserting this module into LocalNO for fitting the NSE improves the rollout error relative to LocalNO alone; for PDE super-resolution, only qualitative results are given; for image super-resolution, it shows some improvement over the backbone alone.

Critical strengths
  1. The core insight synthesized in the paper, namely the observation that many signals have self-similar scales, together with the authors' theoretical, methodological, and engineering contributions, has broad potential impact on the AI community. Most natural signals, even images, exhibit a certain degree of self-similarity. Improving models' ability to learn signals from this perspective is inspiring and practically useful for processing such data.

  2. The idea proposed in the paper of learning these self-similar scales is fairly general. Moreover, among the formalized concrete methods, S3C has a very small parameter count and extra overhead, and yields substantial metric improvements for some models and settings.

Critical weaknesses
  1. The core problem of the paper might already be solvable by existing methods, and the implementation is disconnected from the theory. On the one hand, there are many methods for addressing spectral bias, some of which are also plug-and-play and can likewise improve frequency-domain expressiveness. However, apart from a frequency-loss baseline that performs even worse than the original LocalNO, the authors only discuss these methods, without verifying whether they can already help the model learn self-similar scales and possibly achieve higher accuracy than S3C. On the other hand, the implementations of S3C and S2WC actually follow the scale recursion proposed in Section 2 only to a shallow extent: under the settings in Appendix E.1, there are only about one and a half octaves above the cutoff at 64×6464\times 64 and about two and a half at 128×128128\times 128; the shared MLP iterates only two to three times, and S2WC has only two steps, which is insufficient to test the cross-scale sharing claimed by the theory. Table 6(b) also does not support that dilation is the key; instead, it suggests that the gains very likely come from a generic latent-space high-frequency reweighting module. The paper also does not test whether cross-scale sharing plays a role.

  2. Both the novelty and the scope of applicability of the paper are actually rather narrow. The insight of self-similar scales in data from various fields is not new [1–4]. The main novel contribution of the paper lies in the theory: it gives bandwidth lower bounds for shared cross-scale operators, proves that the dilation ratio must equal the band ratio, and unifies transposed convolutions and fractal coding. However, the individual operations of S3C and S2WC are not new, and more seriously, they implement the scale recursion required by the theory only shallowly in the experiments. In addition, PDE simulation experiments are usually validated on multiple PDEs, e.g., PDEBench contains 11, but the paper's rollout results cover only one setting of one PDE, and clear accuracy improvements appear mainly on a single backbone, LocalNO.

[1] Charles Meneveau et al. Scale-Invariance and Turbulence Models for Large-Eddy Simulation. Annual Review of Fluid Mechanics, 2000.

[2] Nicola Rebora et al. RainFARM: Rainfall Downscaling by a Filtered Autoregressive Model. Journal of Hydrometeorology, 2006.

[3] Geoffrey M. Davis. A Wavelet-Based Analysis of Fractal Image Compression. IEEE Transactions on Image Processing, 1998.

[4] Wei-Sheng Lai et al. Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.

Questions for authors
  1. Could the authors fully implement the scale recursion required by the theory and then demonstrate its contribution to accuracy through ablations?

  2. Could the authors add strong baselines that are likewise designed to improve spectral accuracy?

  3. Could the authors provide rollout results on more PDEs, e.g., using PDEBench?

20 comments

Thanks for your thorough and constructive feedback! We address each of your points and questions below and have updated the manuscript in OpenReview with red text highlighting all additions.

On the one hand, there are many methods for addressing spectral bias, some of which are also plug-and-play and can likewise improve frequency-domain expressiveness. However, apart from a frequency-loss baseline that performs even worse than the original LocalNO, the authors only discuss these methods ...

Could the authors add strong baselines that are likewise designed to improve spectral accuracy?

Thanks for raising this point! As mentioned in Appendix B.2, S3C is intended to be "orthogonal and complementary to the successful prior techniques for mitigating spectral attenuation" and is thus not intended to replace existing modules and solutions. However, we understand that this discussion was in the appendix instead of in the main text, and we have emphasized this perspective more in the experimental section (by adding the new Section 4.4).

In fact, we choose LocalNO as one of the backbones in our paper because its two parallel local integral and differential branches were introduced precisely to combat spectral attenuation caused by FNO's global convolution. Indeed, in our experiments, we show that LocalNO is capable of improving the results on Navier-Stokes rollout compared to FNO, and combined with S3C the performance further improves. S3C is thus complementary to LocalNO as a mechanism of reducing spectral attenuation (Table 1).

Furthermore, we agree that including the frequency loss baseline in the main text is confusing and does not convey this message of complementarity. As such, we have moved it to the appendix and replaced it with a section (4.4) comparing training on the H1H^1 Sobolev loss, which is a plug-and-play approach commonly used in operator learning to better learn high frequencies [1]. The results are shown below with errors reported ×10−2\times 10^{-2}.

Model / MethodStepsL2L^2 ErrorH1H^1 ErrorHF32\text{HF}_{32} Error
LocalNO H116.18.943.3
27.912.550.0
411.720.059.7
617.129.677.4
818.741.999.1
1033.856.1122.6
LocalNO + S3C H114.86.933.5
26.710.440.7
410.116.748.8
613.923.757.8
818.832.369.7
1024.542.683.6
LocalNO L216.19.242.9
28.013.149.0
411.420.157.2
615.127.268.6
818.733.779.8
1022.740.690.3

While training the LocalNO baseline on H1 loss does not strictly outperform training it on L2 loss (without S3C) everywhere, in the few-step regime (we only directly train on 1-step rollouts in this paper), the H1 error of the former model is slightly better than that of the latter. Furthermore, we see that S3C is strictly complementary to H1 loss as a mechanism of ameliorating spectral attenuation. In the new Section 4.4, we show the new H1H^1 results and discuss complementarity of S3C with other methods. See Table 3 in the revised text.

On the other hand, the implementations of S3C and S2WC actually follow the scale recursion proposed in Section 2 only to a shallow extent: under the settings in Appendix E.1, there are only about one and a half octaves above the cutoff at 64 x 64 and about two and a half at 128 x 128; the shared MLP iterates only two to three times, and S2WC has only two steps, which is insufficient to test the cross-scale sharing claimed by the theory.

Thanks for raising this point! We fully agree that it is difficult to isolate the proposed benefit of our framework and architectures in this regime. As such, we present zero-shot super-resolution results for both S3C and S2WC, where we not only apply the shared module to deeper recursions but also at scales not seen during training.

For S3C, we introduce new zero-shot super-resolution results on a new Kuramoto-Sivashinsky (KS) rollout setting (see response to your comment about adding PDE settings below for more details). As identified in your comment above, we find zero-shot super-resolution on Navier-Stokes with S3C to slightly underperform compared to the baseline FNO since learning the evolution map at turbulent Reynolds number 5000 is challenging and S3C only supervised on a few steps struggles in generalizing to unseen bands. However, we find that S3C leads to substantial improvements in zero-shot super-resolution on our new KS equation rollout results.

In particular, we find that for FNO + S3C trained up to resolution 64 (1 step of the recurrent S3C) on the KS equation, zero-shot super-resolution is possible at resolution 256 (7 steps of S3C recurrence; 6 unseen at training time):

Stepsrel. L2L^2 Baserel. L2L^2 +S3Crel. H1H^1 Baserel. H1H^1 +S3Crel. HF$_{32}$ Baserel. HF$_{32}$ +S3C
10.570.271.730.8224.711.5
20.960.392.891.1441.216.1
41.360.454.091.3058.418.4
61.510.474.521.3264.618.7
81.570.494.661.3366.618.7
101.590.524.711.3567.218.7

Please refer to the full KS experimental results below for more information.

We have also devised a method to extend S2WC to scales finer than those seen during training while avoiding visual artifacts while achieving competitive results compared to baseline methods. Specifically, we find that the additional detail added in subsequent S2WC recurrent iterations affects the correctly-predicted ×2\times 2 and ×4\times 4 scales. As such, we propose to use back-projection as a technique to fix the correctly-predicted scales at test-time to ensure that the predicted finer scales (e.g., ×8\times 8) strictly add detail at the respective frequency band. We rely on iterative back-projection [9] to achieve this, which takes the model's trusted prediction (e.g., at ×4\times 4 scale) y^p\hat{y}_p and the model's (possibly erroneous) prediction at the target scale y^\hat y and iterates y^←y^+up(y^p−down(y^))\hat y \gets \hat y + \text{up}(\hat{y}_p - \text{down}(\hat y)) for a small fixed number of iterations (we find that repeating 5 times is sufficient), with respective upsampling and downsampling operations. The goal of this projection is to ensure that the coarse scales of y^\hat y match those of y^p\hat{y}_p, so that y^\hat y contributes only fine-scale content. Our super-resolution results can be found below, where missing entries are not published.

MethodDIV2K-val ×6DIV2K-val ×12Set5 ×6Set5 ×8Set5 ×12Set14 ×6Set14 ×8Set14 ×12B100 ×6B100 ×8B100 ×12Urban100 ×6Urban100 ×8Urban100 ×12Manga109 ×6Manga109 ×8Manga109 ×12
SwinIR-MetaSR26.9423.8029.0927.0224.8226.5825.0923.3325.9424.8723.5924.1622.7521.3127.2924.9622.35
SwinIR-LIIF27.1524.0229.4627.36–26.8225.34–26.0725.01–24.5923.14–27.6925.28–
SwinIR-LTE27.2024.0929.5027.35–26.8625.42–26.0925.03–24.6223.17–27.8325.42–
SwinIR-S2WC (ours)27.0623.8429.2626.9824.7826.6825.0723.2626.0024.8623.5624.4122.8121.3727.4224.9022.52

We would also like to note that architectures like LIIF and LTE and their coordinate MLP output heads are highly-optimized for the arbitrary-scale super-resolution task, whereas S2WC is only one instantiation of our more general framework that already performs competitively, even on unseen scales. In the future, methods like S2WC could also be further optimized to enforce the coarse-scale projection as a self-consistency loss during training, removing the need for the explicit projection step. S2WC super-resolution results have been incorporated into the Appendix in the revised version.

Reviewer_UgswOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes self-similar cascade decoders to mitigate the attenuation of fine-scale components in neural network predictions. It formulates coarse-to-fine synthesis as a scale recurrence with a shared operator and analyzes how coordinate dilation permits frequency-support expansion with a scale-independent kernel bandwidth. Two practical realizations are introduced: S3C, a lightweight Fourier-amplitude cascade that retains the backbone-predicted phases, and S2WC, a wavelet-domain attention decoder. Validation includes turbulent Navier-Stokes autoregressive forecasting, PDE smoothing/spectral extrapolation, and deterministic image super-resolution. The most compelling results are the improvements in fluid forecasting obtained with a small additional parameter budget.

Critical strengths
  1. The central motivation is interesting and practically meaningful, particularly for physical systems such as turbulence, where inaccurate fine-scale predictions can affect subsequent temporal evolution. S3C offers a compact way to incorporate cross-scale structure into existing predictors, and the experiments across several backbones and rollout horizons provide useful evidence of its practical value.
Critical weaknesses
  1. The introduction does not sufficiently focus the target application scope. It motivates the work through a broad range of natural signals and domains, but does not clearly explain which class of practical problems the paper primarily targets or why the selected forecasting and reconstruction tasks are representative. In particular, the connection between fine-scale attenuation and accumulated errors in turbulent autoregressive forecasting deserves a more explicit motivation. The general framing also makes it difficult to distinguish the broad ambition of the framework from the narrower scope established by the experiments.

  2. The incremental contribution and the evidence for the proposed mechanism need sharper positioning. Self-similarity priors, cross-scale attention, and parameter sharing across progressive reconstruction stages have substantial precedents, including Neural Collages (Poli et al., 2022), MS-LapSRN (Lai et al., Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks), and CS-NL (Mei et al., CVPR 2020). The paper discusses some of this lineage, but a more direct comparison would help establish what the new framework and its instantiations add. Moreover, Appendix F/Table 6 shows very similar performance for translational and dilational frequency spacing. This leaves the distinctive empirical benefit of geometric scale recurrence insufficiently isolated. The theory establishes frequency-support/energy-routing properties rather than accurate reconstruction of the true fine-scale coefficients.

Questions for authors
  1. Could the authors state the primary target class of practical tasks more explicitly, and clarify how the turbulence forecasting, PDE smoothing, and image super-resolution experiments support that scope? In particular, which claims concern spectral-amplitude recovery, and which concern recovery of the actual fine-scale structure?

  2. Given the small gap between translational and dilational spacing in Table 6, what evidence isolates the benefit of the proposed self-similar scale recurrence from that of a generic learned spectral-amplitude correction? A parameter-matched comparison with a simple spectral filter or power-law extrapolator, and an explanation of the distinction from prior shared-scale reconstruction methods, would materially clarify my assessment.

10 comments

Thanks for your useful feedback and support on our manuscript! We address the specific weaknesses and questions you mentioned below and have updated the manuscript in OpenReview with red text highlighting all additions.

The introduction does not sufficiently focus the target application scope ... In particular, the connection between fine-scale attenuation and accumulated errors in turbulent autoregressive forecasting deserves a more explicit motivation.

Could the authors state the primary target class of practical tasks more explicitly, and clarify how the turbulence forecasting, PDE smoothing, and image super-resolution experiments support that scope? In particular, which claims concern spectral-amplitude recovery, and which concern recovery of the actual fine-scale structure?

Thank you for raising this point. We agree that in the initial draft, the introduction did not explicitly mention the primary target class of practical problems. We have clarified this in the revised introduction. Specifically, our theoretical framework and practical methods are applicable when neural networks struggle to model or output self-similar signals. Intuitively, these are signals whose finer scales are determinable from their weak ones. This has been well-documented in many domains, several of which are listed in the introduction, including natural images, turbulent fluids, waves, brain activity, etc. While in principle our methods could be applied to domains such as these that pervade the natural sciences, we focus on natural images and PDE forecasting because they are of general interest to the machine learning and applied mathematics communities, and because there are established benchmarks and datasets for both. For instance, spectral attenuation has been long observed in computer vision applications, and attempts to correct it have been explored perhaps most famously by the neural field literature (e.g., SIREN, Fourier features). We focus on image super-resolution as perhaps the paradigmatic such problem in vision because the task itself involves the creation of finer detail from coarser scales. Super-resolution has been explored in-depth in the literature, but less so from the angle of self-similarity, which we take in this paper. We focus on turbulent fluid flow forecasting because the theory surrounding its spectral properties is more rigorously established than for natural images. For instance, in 2d flows, it is known that energy cascades from fine scales to coarse scales [4], causing errors at finer scales to propagate to coarser ones [5] (this explains why S3C's improvements at 1-step prediction compound over many steps). Classical subgrid-scale modeling has also shown that excessive dissipation of resolved small scales can bias the large-scale flow [6]. With important relevance to climate and weather modeling, we focus on turbulent flow forecasting as another target domain.

Self-similarity priors, cross-scale attention, and parameter sharing across progressive reconstruction stages have substantial precedents ... a more direct comparison would help establish what the new framework and its instantiations add.

Thanks for these references! We have added the latter two to and expanded the discussion on the former in our revised Appendix B.1. The neural collage framework [1] is a differentiable and learned version of classical fractal image coding. We discuss it in B.1 and prove in Section 2.3 that their work is a special case of our general framework. Conversely, our work can also be seen as a generalization of [1] to function spaces and arbitrary bases and approximation spaces.

MS-LapSRN [2] introduces a progressive coarse-to-fine super-resolution approach with parameters shared across scales, where each level predicts a residual and uses transposed convolution to upsample. In Section 2.3 (and proved in the appendix), we discuss how transposed convolutions from deep learning are a special case of our framework. As such, MS-LapSRN can be similarly seen as an instantiation of our cascade decoder family, instantiated in the zero-spline basis. Our own S2WC is an instantiation using local attention and in the wavelet basis.

CS-NL [3] introduces cross-scale non-local attention between latent features and their downsampled counterparts. This is similar to the cross-scale attention used in our proposed S2WC architecture. However, CS-NL does not implement a cross-scale cascade and weight sharing across scales, so it does not fit precisely under our framework.

Given the small gap between translational and dilational spacing in Table 6, what evidence isolates the benefit of the proposed self-similar scale recurrence from that of a generic learned spectral-amplitude correction?

Thanks for raising this point! We would first like to clarify the theoretical distinction between dilational and translation spacing, which was not made very clear in our first draft. Formally, Theorem 1 is a statement about the support of Sϕ^\widehat{S\phi}; the corollary about λ=γ>1\lambda = \gamma > 1 holds true for the choice of the bands BkB_k introduced in Section 2.2. We focused on such geometrically-spaced Fourier bands because they are well-suited to the study of power law decays, which Navier-Stokes and natural images have been classically known to exhibit. In such cases, our theory predicts that dilational spacing should outperform translational spacing (see below for updated ablations on this). However, in the case of exponential decay, for instance, fixed-width Fourier bands may be a more suitable choice for analysis. In these cases, such as for the Kuramoto-Sivashinsky PDE, we would expect translational (fixed-width) spacing to outperform dilational spacing. We ablate both of these in the results that follow.

Since this was unclear in the original draft, we have updated Section 2.2 and the informal statement of Theorem 1 to reflect this distinction. We have also added an Example 2 on learning an exponential decay accompanying the existing Example 1 for learning a power law decay. Please refer to the revised main text for details.

Next, we would also like to make a minor correction to our comparison between dilational and translational spacing. In the initial version of our Table 6, at training resolution dilational vs. translational performed approximately the same, within the variance threshold between runs. We have since found that this occurs because at the 64×6464 \times 64 training resolution, the band spacing of dilation and translation are nearly identical. As such, they perform very similar operations when evaluated at training resolution. However, at unseen higher resolutions, the band spacing of translation and dilation no longer coincide. As such, we ablate difference between dilation and translation spacing for Navier-Stokes on FNO trained at 64×6464 \times 64 and evaluated zero-shot at 128×128128 \times 128. Since Navier-Stokes exhibits a power law decay, a dyadic decomposition (BkB_k as defined in Section 2) is well-suited to model its spectrum. As such, we expect dilational spacing to outperform translational spacing on the unseen resolutions, which is indeed what we observe:

Stepsrel. L2L^2 Dilationalrel. L2L^2 Translationalrel. H1H^1 Dilationalrel. H1H^1 Translationalrel. HF$_{32}$ Dilationalrel. HF$_{32}$ Translational
121.722.455.257.099.0102.3
224.324.960.963.0103.5107.1
430.431.172.875.2112.1116.2
639.040.089.292.1122.6127.7
848.950.2106.3109.6131.7138.0
1058.760.1120.4124.0137.7144.7

Furthermore, we use this zero-shot super-resolution setting to compare with a fixed power law extrapolation and with a per-shell MLP (parameter-matched), following your recommendation. Compared to S3C, all three methods perform similarly at the seen resolutions, which supports our claim that S3C helps learn the power law spectral decay in this setting. However, S3C outperforms both on our zero-shot super-resolution ablation.

Stepsrel. L2L^2 S3Crel. L2L^2 Per-shell filterrel. L2L^2 Power lawrel. H1H^1 S3Crel. H1H^1 Per-shell filterrel. H1H^1 Power lawrel. HF$_{32}$ S3Crel. HF$_{32}$ Per-shell filterrel. HF$_{32}$ Power law
121.749.522.555.292.257.299.0224.3102.4
224.3diverges25.160.9diverges63.2103.5diverges106.9
430.4diverges31.972.8diverges76.8112.1diverges117.8
639.0diverges41.489.2diverges95.0122.6diverges131.1
848.9diverges51.8106.3diverges112.5131.7diverges141.6
1058.7diverges61.4120.3diverges125.6137.6diverges147.4

We parameterize the per-shell MLP by passing in the normalized target wavenumber, but find that this parameterization is substantially less stable at out-of-domain extrapolation compared to S3C.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.