acceptodds
Under review as a conference paper at ICLR 2027

Align the Skip: Spatially Aligned and Temporally Resolved Hierarchical Surrogates for Multiscale Turbulence

Abstract

Hierarchical encoder-decoder surrogates efficiently represent multiscale turbulent dynamics, but their skip connections fuse input-time encoder features with future-oriented decoder features at matching grid indices. Under advection, these features need not remain spatially correspondent. We introduce decoder-predicted spatial alignment, which learns where encoder features should be sampled before skip fusion. Across three fluid and plasma benchmarks, alignment substantially improves autoregressive forecasting and supports accurate prediction of multiple states within a larger forecast interval, reducing complete autoregressive model calls while preserving intermediate-time outputs. These intermediate states also partition each outer interval into shorter subintervals for physics-informed training through equation supervision. At a fixed outer interval, increasing the number of internal subintervals from one to eight reduces the 64-step Kolmogorov-flow relative error by more than 40%, bringing equation-supervised performance close to data-supervised accuracy without using future-state targets. We further train full-resolution surrogates using spatially reduced observations of the training initial states together with a persistent prediction-derived unresolved state. This state is iteratively constructed from model predictions and carried across optimization, allowing initially absent fine-scale structure to emerge and adapt. Reducing the effective training observations from 64x64 to only 2x2 block averages, a 1024x reduction in observed spatial values, increases the 64-step error by just 17% when evaluation begins from full-resolution held-out states. Even when the observed training states contain no spatial variation, small white noise seeds unresolved structure that is subsequently developed through equation supervision and persistent model-generated state. These results show that explicit spatial correspondence and temporally resolved equation supervision improve multiscale forecasting while reducing reliance on paired current-to-future states and spatially resolved training initial conditions.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural PDE surrogates, physics-informed machine learning, hierarchical encoder-decoder networks, equation-supervised learning, learned spatial alignment, autoregressive forecasting, multiscale turbulence, neural operators

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_93WYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents a decoder predicted skip alignment method for improving surrogate models for PDE forecasting tasks in advective systems. The proposed method has each decoder level predict a learned displacement field that resamples the corresponding encoder skip to align with known movement that happens as time marches forward in the advective PDE system. The main contribution is the development of the skip connection alignment methodology for multiscale encoder decoder architectures along with the extension of the methodology to be used in multi-step and multi-state scenarios, which allow for more accurate predictions and fewer model invocations for long horizon rollouts. The method is shown to work in cases where target data is used as well as for cases where a physics loss is use instead when the multi-time step scenarios do not align with target data. This methodology is also used to show that under physics supervision the model can be trained from coarse inputs to still forecast accurately from resolution states.

The authors present various experiments evaluating the alignment methodology on various advective PDE flows showing improved accuracy over various scenarios. These experiments inlcude baseline comparison, ablation studies, use with different multiscale convolutional encoder decoder architectures, testing of accuracy in multi-step and multi-state scenarios, use of physics loss for data free training, and transfer from coarse inputs to higher resolution states.

One critique is there a multitude of experiments testing the three different contributions. The main text relies heavily on the appendix for presenting results making it hard for the reader to follow and understand the significance of the different results.

Critical strengths

The consistent benefit of the alignment across a broad range of settings.

Critical weaknesses

The presentation of the results is difficult to follow. The main text is not self-contained relying heavily on the appendix.

Questions for authors

What is the practical utility of the method as a simulation surrogate? How would it be used in realistic scenarios?

Reviewer_4uE9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes 3 ideas for improving PDE surrogate models: (1) add a learnable spatial offset to the skip connections of U-Net-style architectures (inspired by advection); (2) for equation-based learning, have the model output intermediate time points in addition to the final output and supervise the residual on these intermediate states; (3) use and update a hidden fine state to provide more diverse initial conditions (also in an equation learning context).

Critical strengths

The paper presents a few interesting ideas, with the advection-inspired skip connections being the most methodologically interesting.

Critical weaknesses
  1. Overall, the paper reads like 3 separate and distinct ideas squeezed into one paper. The alternative skip connections are conceptually interesting, but it is not clear what role they are playing, especially given the additional complexity. The temporally resolved supervision reads more like a training optimization rather than novel methodology. Finally, the spatially reduced observations are a very convoluted way of getting additional diversity in the distribution of initial conditions. I would suggest that the authors focus on the first contribution. The latter two are much weaker.
  2. Baselines are severely lacking. The skip connections should be compared with other methods of including advection-inspired components (e.g., https://arxiv.org/abs/2610.05098). The temporally resolved supervision should be compared with simply integrating the PDE over the same time window. The reduced observations should be compared with simply generating diverse initial conditions by running a PDE solver or even rolling out the model itself.
Questions for authors
  1. Can you interpret the role that your alternative skip connection is playing? Is it really representing advection in some way, and can it be demonstrated in your model?
  2. Why use such a convoluted process for generating initial conditions when there are many methods available for generating them? Including the simple baselines described above, what about methods like active learning/rare-event sample etc.?
6 comments

We thank the reviewers for their feedback and comments, which help us improve the paper. The reviewer raised concerns regarding the novelty, value, coherence, soundness, and justification of the paper’s contributions. Before addressing these concerns individually, we would like to clarify the broader perspective and practical motivation behind this work.

Our research spans machine learning, numerical methods, turbulence modeling, and physics. We therefore approach surrogate modeling not only as a problem of neural-network design, but also from the practical perspective of modeling physical systems when numerical or experimental data may be limited, corrupted, incomplete, or available only at low spatial resolution. Our goal was to develop simple and computationally efficient modifications to existing approaches that enable accurate surrogate models to be trained and used under these challenging data conditions.

Although our methods build on established ideas in surrogate modeling, learned spatial warping, and equation-supervised training, we respectfully disagree that they merely combine previously explored techniques. To the best of our knowledge, the specific methods proposed in this work are novel. Despite their relative simplicity, they produce substantial improvements in predictive accuracy for convection-dominated systems. Importantly, these improvements are not limited to a single selected model size or prediction interval. Our experiments demonstrate systematic improvements in the accuracy-computational-cost trade-off across multiple model sizes and prediction intervals, supported by controlled ablations, comparisons with closely related architectures, and physical analyses of the learned representations.

The three contributions are also important as a unified methodology. They address complementary obstacles to training accurate and efficient surrogate models from limited data, and they reinforce one another. Skip alignment improves hierarchical prediction across nonzero temporal intervals. This enables more accurate intermediate-state prediction and fractional-step equation supervision without future-state targets. Persistent-state training then uses this temporally refined, equation-supervised framework to develop physically consistent full-resolution training conditions when only severely reduced spatial observations are available. Together, these contributions address the difficult and practically important problem of learning accurate, computationally efficient, full-resolution dynamics from progressively less complete training data.

In our subsequent responses, we address the specific points raised by the reviewers.

AuthorsOpenReview ↗

Reply to Reviewer 4uE9 regarding the role that the alternative skip connections are playing

Reviewer 4uE9: Can you interpret the role that your alternative skip connection is playing? Is it really representing advection in some way, and can it be demonstrated in your model?

We thank the reviewer for this question. New Appendix J directly tests whether the learned alignment is related to physical advection. For both Kolmogorov flow and Hasegawa-Wakatani turbulence, we measure the learned displacement at every skip resolution and for prediction horizons nt∈{1,2,4,8}n_t \in \{1,2,4,8\}. We compare it with a finite-time advective displacement obtained by integrating ground-truth characteristics backward over the same interval. This convention matches the backward sampling used by the learned feature warp.

For Kolmogorov flow, where vorticity uniquely determines the advecting velocity, the finest-resolution learned displacement closely matches the physical reference. The learned-to-reference displacement-magnitude ratios are approximately 1.021.02, 0.990.99, 0.980.98, and 0.960.96 for nt=1,2,4,n_t=1,2,4, and 88, respectively. The learned shifts become smaller at deeper resolutions, showing that the network does not simply reproduce the same physical flow map at every level. Instead, it learns resolution-dependent correspondence for increasingly coarse and transformed latent features. This observation supports predicting displacement independently at all skip levels, especially for longer transitions, consistently with the ablations in Appendix H.3.

For Hasegawa-Wakatani turbulence, the equation-derived reference has a less direct relationship to the complete learned state. The potential-derived E×BE\times B drift advects density nn and vorticity ω=∇⊥2ϕ\omega=\nabla_\perp^2\phi, while the network predicts (n,ϕ)(n,\phi). The potential ϕ\phi is related nonlocally to ω\omega and is not directly self-advected. The channel-shared learned shifts are therefore lower than the drift reference, but remain of the same order, reaching approximately 55%55\% to 65%65\% of its magnitude at the finest level. This behavior is consistent with a latent correspondence field that must serve coupled features with different transport behavior. It also motivates the per-channel alignment used in our principal model.

Thus, the learned alignment is quantitatively related to advection, most directly for Kolmogorov flow, while remaining a resolution- and representation-dependent latent correspondence mechanism rather than an explicitly constrained physical flow map.

AuthorsOpenReview ↗

Reply to Reviewer 4uE9 regarding the proposed method for generating initial conditions

Reviewer 4uE9: Why use such a convoluted process for generating initial conditions when there are many methods available for generating them? The reduced observations should be compared with simply generating diverse initial conditions by running a PDE solver or even rolling out the model itself.

We thank the reviewer for this question. The purpose of our procedure is more specific than generating diverse inputs: it is to generate diverse, physically plausible full-resolution initial conditions when only low-spatial-resolution training data are available. In the manuscript, we refer to each such low-resolution training field as a reduced observation. These fields lack the high-frequency content needed to represent developed turbulence. Interpolation restores the full grid size, but not the missing spatial scales.

Mechanistically, each low-resolution training field is a separate training example. Once it is processed by the model, we extract from its prediction the high-frequency component that is missing from the low-resolution input and store it as an example-specific persistent state. When the same example is encountered again, its persistent state is added to the interpolated low-resolution field to construct an updated full-resolution initial condition. Each new prediction then updates the missing fine-scale component. Consequently, the generated initial conditions differ across examples and continue to evolve between epochs instead of being independently resampled. Our method therefore already implements the reviewer's suggestion of rolling out the model, but maintains a separate persistent rollout for every training example. The update is defined in Eq. (6) and described in Sections 3.4 and 4.5 and Appendix G.

Despite its simplicity, this persistent model-evolution procedure is highly effective. To the best of our knowledge, prior work has not demonstrated training a full-resolution turbulent-flow surrogate from spatially uniform 1×11\times1 training observations, with no resolved spatial input from the PDE solver. In this limiting case, the model develops the entire spatially varying component from small symmetry-breaking perturbations through persistent, equation-constrained evolution. The resulting fields and their spectra are examined in Appendix H.8.2.

The physical motivation is that small perturbations in an unstable system initially grow and eventually reach a nonlinear saturated regime. We emulate this development during training. In the limiting 1×11\times1 case, the reduced observation contains only the spatial mean, and small decaying random perturbations are used to break spatial symmetry. Subsequent fine-scale structure is generated through repeated equation-constrained model evolution. At higher observation resolutions, the generated unresolved component is also conditioned on the distinct resolved structure of each observation.

The objective is not diversity alone, but initial conditions consistent with the physical distribution of developed turbulence. We examine this in Appendix H.8.2, where the generated states develop broadband spectra characteristic of the full-resolution system. The spectral analysis therefore tests whether the generated conditions contain physically appropriate scale distributions rather than arbitrary high-frequency variation. In machine-learning terms, the method generates samples conditioned on the observed coarse state while approaching the distribution of developed full-resolution states.

We compare against both relevant alternatives. First, the full-resolution-input baseline uses diverse initial conditions produced directly by the PDE solver. The reduced-observation experiments are obtained by spatially averaging these states from 64×6464\times64 down to 1×11\times1, allowing us to isolate the effect of withholding their fine-scale information. These results are reported in Section 4.5, Fig. 3(b), and Appendix H.8.1.

Second, Appendix H.8.3 and Table 6 compare persistence with independently resampled Gaussian perturbations. Fresh noise also creates diverse inputs, but does not preserve and develop an example-specific turbulent state. At 64×6464\times64 observation resolution, the best fresh-noise control has a seed-mean 64-step error of 0.24430.2443, compared with 0.10210.1021 for the persistent-state model. At 8×88\times8, fresh-noise controls exhibit strong seed-dependent failures, whereas the persistent-state model obtains 0.1148±0.02990.1148\pm0.0299. The relevant distinction is therefore whether the missing scales are persistently evolved toward a dynamically and statistically appropriate state. We will clarify that the method generates diverse, observation-conditioned full-resolution initial conditions whose unresolved components evolve across epochs. Their physical plausibility is assessed through their spectra, and their utility is compared with both PDE-generated full-resolution states and freshly resampled perturbations.

Reviewer_UPzAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposed three methods to improve learning turbulence dynamics. First, the authors introduced an "alignment" step within the decoder that resamples the output grid based on a learned displacement. Next, the authors proposed to predict multiple intermediate time steps in order to accurately calculate PINN loss to help regularizing the model. Finally, the authors experiment with learning from downsampled inputs, by accumulating the model output over steps.

Critical strengths

This paper proposed a simple warp-based augmentation that consistently improves the predictive performance of fluid surrogates. The authors coupled it with PINN loss to further improve its accuracy. The low-resolution example shows that the PINN loss is able to reduce the error even when there's little spatial information.

Critical weaknesses
  1. Warp-based model is not new to this field, and the proposed architecture is heavily motivated by prior works like flowers and ADA-CNNs. The authors reported these related works but never compare to them / ablate their design choices.
  2. This paper consists of three loosely coupled methods. Each of them stands on its own and has already been explored in some ways by this community. The authors only demonstrate their performance against old baselines on simple 2D datasets, which can't demonstrate the usefulness of all these proposed methods on challenging problems.
Questions for authors
  1. Can you clarify the advantages of the proposed design versus prior warp-based models?
  2. Why using temporal discretization = 16 makes error higher? Does it imply some hidden assumptions not being clarified?
3 comments

We thank the reviewers for their feedback and comments, which help us improve the paper. The reviewer raised concerns regarding the novelty, value, coherence, soundness, and justification of the paper’s contributions. Before addressing these concerns individually, we would like to clarify the broader perspective and practical motivation behind this work.

Our research spans machine learning, numerical methods, turbulence modeling, and physics. We therefore approach surrogate modeling not only as a problem of neural-network design, but also from the practical perspective of modeling physical systems when numerical or experimental data may be limited, corrupted, incomplete, or available only at low spatial resolution. Our goal was to develop simple and computationally efficient modifications to existing approaches that enable accurate surrogate models to be trained and used under these challenging data conditions.

Although our methods build on established ideas in surrogate modeling, learned spatial warping, and equation-supervised training, we respectfully disagree that they merely combine previously explored techniques. To the best of our knowledge, the specific methods proposed in this work are novel. Despite their relative simplicity, they produce substantial improvements in predictive accuracy for convection-dominated systems. Importantly, these improvements are not limited to a single selected model size or prediction interval. Our experiments demonstrate systematic improvements in the accuracy-computational-cost trade-off across multiple model sizes and prediction intervals, supported by controlled ablations, comparisons with closely related architectures, and physical analyses of the learned representations.

The three contributions are also important as a unified methodology. They address complementary obstacles to training accurate and efficient surrogate models from limited data, and they reinforce one another. Skip alignment improves hierarchical prediction across nonzero temporal intervals. This enables more accurate intermediate-state prediction and fractional-step equation supervision without future-state targets. Persistent-state training then uses this temporally refined, equation-supervised framework to develop physically consistent full-resolution training conditions when only severely reduced spatial observations are available. Together, these contributions address the difficult and practically important problem of learning accurate, computationally efficient, full-resolution dynamics from progressively less complete training data.

In our subsequent responses, we address the specific points raised by the reviewers.

AuthorsOpenReview ↗

Reply to Reviewer UPzA Regarding Ablation of design choices for the aligned U-Net models

We thank the reviewer for raising the point regarding ablating the design choices. We respectfully note that the manuscript already includes controlled ablations of both the presence of skip alignment and its principal design choices. Matched aligned and unaligned convolutional U-Nets are compared across model widths and outer prediction intervals in Section 4.2 and Appendices H.2 and H.4. These models use the same width, hierarchy depth, optimizer, learning-rate schedule, data partition, and random seeds, and differ only in whether the encoder skips are spatially aligned before decoder fusion. Across the reported comparisons, alignment consistently improves the prediction accuracy versus training-cost trade-off, with the advantage generally increasing over longer autoregressive rollouts. For example, at nt=1n_t=1 (shortest prediction time step), alignment reduces the 64-step Kolmogorov-flow seed-mean error by factors of approximately 4.0, 3.0, and 2.7 for widths 16, 32, and 64, respectively; the 16-channel aligned model also outperforms the 64-channel unaligned model.

Appendix H.3, Fig. 6, and Table 2 further ablate three specific alignment choices on both Kolmogorov flow and Hasegawa-Wakatani turbulence: (i) predicting displacement from the future-oriented decoder rather than from the input-time encoder, (ii) predicting an independent displacement for each latent channel rather than sharing one field across channels, and (iii) aligning every skip resolution rather than only the finest skip. Decoder-predicted alignment is the dominant choice, reducing the seed-mean error relative to encoder-predicted alignment by 42% to 71% for Kolmogorov flow and 18% to 25% for Hasegawa-Wakatani across the tested intervals. Per-channel displacement gives a smaller but consistent improvement at every tested interval. Extending alignment from the finest skip to all resolution levels is more interval dependent: the finest-level variant can be sufficient for short transitions, whereas all-level alignment improves accuracy for longer transitions, most clearly at nt=8n_t=8 for Hasegawa-Wakatani turbulence. Overall, the decoder-predicted, per-channel, all-level design gives the lowest seed-mean error at every tested interval in both systems. These experiments support the selected design while also showing which components account for its improvement.

AuthorsOpenReview ↗

Reply to Reviewer UPzA regarding demonstrating model performance beyond 2D datasets

Reviewer UPzA: The authors only demonstrate their performance on simple 2D datasets, which can't demonstrate the usefulness of all these proposed methods on challenging problems.

Thank you for raising this concern. We have added experiments on forced three-dimensional homogeneous isotropic turbulence, using the dataset introduced by (Li et al., 2023), in the new Appendix K. This benchmark evolves pressure and all three velocity components on a 60360^3 periodic grid according to the forced incompressible Navier--Stokes equations. The external forcing maintains statistically stationary turbulence, while the three-dimensional dynamics include vortex stretching, making this a substantially more challenging test than our original 2D systems.

We compare aligned and unaligned 3D U-Nets with three resolution levels at two model widths and two prediction intervals, using three seeds per configuration. Alignment improves the seed-mean eight-step relative L2L_2 error in every matched-width comparison. Moreover, the 16-channel aligned model slightly outperforms the 32-channel unaligned model at both prediction intervals while requiring approximately 42% less training time. These results demonstrate that alignment improves the accuracy-to-cost trade-off on a challenging three-dimensional turbulent system and that its benefit is not limited to the original 2D benchmarks. Appendix K provides the governing equations and complete dataset, model, training, and evaluation details, together with the corresponding figure.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Reply to reviewer UPzA and 4uE9 remarks regarding three separate contributions

Reviewer UPzA: This paper consists of three loosely coupled methods. Each of them stands on its own and has already been explored in some ways by this community.

Reviewer 4uE9: The paper reads like 3 separate and distinct ideas squeezed into one paper. I would suggest that the authors focus on the first contribution. The latter two are much weaker.

We agree that the three components are modular and can be studied independently. However, they are not loosely connected additions. They address three distinct obstacles to learning accurate, temporally resolved PDE surrogates under progressively weaker data assumptions. Skip alignment corrects the spatial mismatch between input-time encoder features and future-oriented decoder features. Intermediate predictions within one outer forecast interval enable more accurate equation-supervised training without future-state targets. Persistent-state training constructs physically plausible unresolved content when only low-spatial-resolution training fields are available. Together, these components support the paper’s central goal of learning full-resolution dynamics at requested times while reducing reliance on densely resolved training trajectories.

As detailed in our separate replies, “Regarding the novelty of skip alignment and comparisons with the closest architectures” and “Ablation of design choices for the aligned U-Net models,” the first contribution is not warping in general. It is decoder-conditioned, channel-specific alignment of temporally mismatched skip features within a hierarchical forecasting model. The controlled experiments in Section 4.2 and Appendices H.2–H.4 isolate the effects of alignment and its principal design choices. The new comparisons with FLOWERS, ADRNet, and SineNet in Appendix I further establish the architectural distinction of our approach and its substantially stronger accuracy-cost trade-off.

Second, we do not claim that queried-time conditioning or direct multi-state output is novel by itself, as stated explicitly in Section 2 and Appendix B.3. The methodological contribution is showing that an aligned hierarchy can efficiently construct multiple intermediate states within one outer prediction interval, and that adjacent predictions can be used to evaluate the PDE residual over shorter fractional intervals. This temporal refinement substantially improves equation-supervised training without requiring paired current-to-future training fields. Moreover, the alignment ablation shows that these gains depend strongly on spatial correspondence: the aligned model benefits from increasing temporal refinement, whereas the matched unaligned U-Net deteriorates when required to construct longer sequences of intermediate states. These results are presented in Sections 4.3 and 4.4 and Appendices H.5 and H.6.

Third, persistent-state training extends the same target-free framework to a substantially weaker information regime. Its purpose is to generate diverse, physically consistent full-resolution training conditions from severely reduced current-state observations, including the 1 x 1 limit in which each supplied field contains only its spatial mean and no spatially varying information from the PDE solver. Each reduced observation maintains a separate prediction-derived unresolved state that evolves across epochs under equation supervision. Temporally refined prediction is important in this regime because it gives the missing spatial scales modeled physical time to develop while still allowing the PDE residual to be evaluated over short fractional intervals.

To the best of our knowledge, training a full-resolution turbulent-flow surrogate in this mean-only regime, without future-state targets or spatially resolved training inputs, has not previously been demonstrated. Sections 3.4 and 4.5, Eq. (6), Appendix G, and Appendices H.8.1–H.8.3 define the procedure, compare it with training from full-resolution PDE-generated states and with fresh-noise controls, and evaluate the multiscale physical consistency of the generated training conditions through spectral analysis.

Thus, the paper does not present three unrelated techniques as individually unprecedented. Its methodological progression is: alignment enables accurate multi-state hierarchical prediction; the resulting intermediate states provide efficient fractional-step equation supervision without future-state targets; and this target-free, temporally refined framework develops persistent unresolved states when spatially resolved current-state inputs are also withheld.

We will revise the manuscript to make this unifying structure and the individual novelty of each contribution clearer. Their modularity is intentional: it enables controlled ablation and independent reuse, while their combination supports the reduced-information, temporally resolved forecasting setting considered in this work.

AuthorsOpenReview ↗

Reply to reviewer UPzA and 4uE9 remarks regarding the novelty of skip alignment and comparisons with the closest architectures

Reviewer UPzA : Warp-based model is not new to this field, and the proposed architecture is heavily motivated by prior works like flowers and ADA-CNNs. The authors reported these related works but never compare to them

Reviewer 4uE9: Baselines are severely lacking. The skip connections should be compared with other methods of including advection-inspired components

Our contribution is decoder-conditioned alignment of temporally mismatched encoder–decoder skips in hierarchical PDE forecasting. At each resolution, the future-oriented decoder predicts channel-specific displacements that spatially align the input-time encoder features before fusion. Conventional convolutions or shifted-window attention remain the within-stage processing operations. To our knowledge, this specific use of learned warping has not been explored in prior PDE forecasting architectures. Appendix B.2 discusses its relationship to earlier feature-alignment and transport methods.

The closest methods address transport differently:

  • ADRNet (Advection Augmented CNNs; Zakariaei et al., 2024) evolves hidden features through updates inspired by reaction–advection–diffusion decomposition, with learned warping providing the advection component. Its transport core consists of sequential blocks operating on full-resolution feature maps, making it structurally closer to a full-resolution residual network than to a multiresolution U-Net. The warp advances the hidden state rather than aligning encoder features with a receiving decoder representation.

  • FLOWERS (Muser et al., 2026) uses multihead warping as the principal within-stage spatial mixer in a U-Net-like hierarchy, while retaining pointwise channel mixing and convolutional resolution changes. Learned displaced sampling therefore performs spatial mixing throughout the hierarchy. Our model retains conventional within-stage processing and applies warping specifically to correct the temporal correspondence mismatch at skip fusion.

  • SineNet (Zhang et al., 2024) identifies temporal skip misalignment and mitigates it by distributing latent evolution across sequential U-shaped waves. Our method directly learns spatial correspondence at each skip within a single encoder–decoder hierarchy.

We will also cite Advectra (arXiv:2610.05098), first posted on October 4, 2026, after the September 25 submission deadline. We could not locate an official public implementation when preparing this response and therefore have not included an experimental comparison. Following the reviewers’ suggestions, we added comparisons with adapted official implementations of FLOWERS, ADRNet, and SineNet. Appendix I, Figs. 20–22, reports model-size sweeps on Kolmogorov flow and Hasegawa–Wakatani turbulence, plotting prediction error against measured training time. For convenience, the tables reproduce representative operating points, including unaligned U-Net as a reference. Entries are seed-mean relative L2L_2 errors at a fixed horizon of 16 stored-data intervals; each model invocation advances ntn_t intervals. Training times are in hours, with ranges across ntn_t. Actual costs are reported rather than assuming exactly matched time budgets; ch denotes hidden width and w the number of waves.

Kolmogorov flow

ModelTraining hnt=1n_t=1nt=2n_t=2nt=4n_t=4nt=8n_t=8
Aligned U-Net (ch=32)1.16–1.170.00900.00850.01100.0143
Unaligned U-Net (ch=32)0.970.01350.01580.01440.0201
FLOWERS (ch=128)1.290.01920.01500.02030.0266
SineNet (w=2)1.69–1.710.03990.01750.01830.0230
ADRNet (ch=32)1.75–1.760.26070.15000.12000.1290

Hasegawa–Wakatani turbulence

ModelTraining hnt=1n_t=1nt=2n_t=2nt=4n_t=4nt=8n_t=8
Aligned U-Net (ch=32)0.52–0.530.14900.11170.10260.1311
Unaligned U-Net (ch=32)0.410.16820.13990.13410.1548
FLOWERS (ch=128)0.58–0.590.32990.26680.23280.2475
SineNet (w=2)0.82–0.830.27640.21420.18290.1878
ADRNet (ch=16)0.64–0.650.50760.39650.31030.2759

Despite its comparatively simple design, aligned U-Net achieves the lowest mean error in all eight displayed comparisons while training faster than the displayed FLOWERS, SineNet, and ADRNet configurations. At nt=4n_t=4, its Kolmogorov error is approximately 46%, 40%, and 91% lower, respectively; the corresponding reductions on Hasegawa–Wakatani are 56%, 44%, and 67%. These substantial gains therefore come with lower training cost.

The complete size sweeps support the strongest overall accuracy–training-cost trade-off for aligned U-Net across the tested configurations. Thus, the proposed skip-alignment mechanism is both conceptually distinct from the closest alternatives and demonstrably effective: a targeted modification to the hierarchy delivers substantial accuracy gains without requiring additional complete U-shaped waves or replacing conventional within-stage processing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.