acceptodds
Under review as a conference paper at ICLR 2027

PACE-FNO: Physics-Aligned Canonical Equivariance for Fourier Neural Operators

Abstract

Neural operators are often tested on states that differ physically from training data. A distinct failure occurs when the physical dynamics are unchanged but the observed coordinate frame differs from training. PACE-FNO addresses this case by estimating the frame, predicting after pulling the field to a canonical representative, and restoring the requested terminal frame. The default inference path uses one forward prediction; optional test-time adaptation (TTA) updates only the low-dimensional coordinate. On translated and Galilean-shifted Burgers and shallow-water systems, PACE-FNO lowers out-of-distribution (OOD) relative error by up to relative to a data-augmented Fourier Neural Operator (FNO+Aug). A matched-estimator control supports attributing the gain to prediction in the canonical frame rather than added estimator capacity. Our analysis separates the canonical approximation error from the two alignment residuals. We also test regional WeatherBench2 time-series forecasting with the fifth-generation ECMWF reanalysis (ERA5) under natural time OOD, where PACE-FNO and FNO achieve nearly identical performance. Experiments with approximate rotation, other backbones, irregular domains, rollouts, and image data delineate the conditions under which the mechanism is effective.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operator, symmetry/equivariance, geometric OOD, PDE/scientific ML

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QYkTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose to wrap a neural operator with input canonicalization and output frame restoration. It learns spatial alignment, computes background velocity analytically, and uses the known dynamics to determine the terminal transformation. Further, test-time adaptation refines the spatial coordinates while keeping network weights fixed. The paper provides a conditional error decomposition and evaluates several PDE benchmarks.

Critical strengths

I found the construction to be sensible. Separating learned spatial alignment from analytically available background velocity and restoring the time-dependent terminal frame has a clear interpretation.

The paper clearly states limitations. For instance, the paper acknowledges worse Burgers ID accuracy, little additional benefit from SWE adaptation, uncertain NS improvements etc.. It also distinguishes approximate arbitrary-angle rotations from exact periodic translation actions.

Critical weaknesses

It's not clear how much benefit comes from learned spatial canonicalization beyond analytic Galilean correction. For Burgers, background velocity is extracted directly from the input mean. This gives the method access to an exact physical correction that the principal FNO baselines do not explicitly implement.

As far as I understand, the paper’s primary contribution is its particular combination of amortized estimation, analytic kinematic reduction, and spectral actions. The paper should demonstrate which components produce an advance over a straightforward learned canonicalizer supplied with the same physical information. It's not clear from the current set of experiments.

The LieLAC comparison is useful for inference cost. However, comparing one-shot prediction with iterative optimization does not isolate the contribution relative to existing amortized canonicalization.

Questions for authors

See above weaknesses

3 comments

Part 1/2 — W1 and W2

Thank you, Reviewer QYkT, for the careful reading. The concerns center on isolating what each component of the wrapper contributes, and the revision adds the corresponding controls: a matched control that disables only the field actions, a matched auxiliary-estimator control for the FNO, and a unified component sensitivity table (Section 5.3, lines 429–446; Appendix Tables 7 and 16).

W1. Benefit of learned spatial canonicalization beyond the analytic correction

The learned coordinate and the analytic correction play different roles: the background velocity is computed analytically from the observed field (Section 3.2, lines 218–225) and enters the terminal coordinate, which still takes the learned spatial coordinate as its input. The revised manuscript adds the controls that isolate the learned part with the analytic reduction in place. In short, the learned coordinate contributes once it is applied: disabling the pullback and pushforward returns OOD error to the 0.630.63–1.041.04 range across the four benchmarks, whereas applying it yields 0.0270.027–0.280.28 (Appendix Table 7, lines 1678–1695; the component-level decomposition is given under W2 below).

The fairness concern is well taken, and the revision adds the corresponding control. FNO+AuxEstimator (Appendix Table 16, lines 2160–2168) trains the same FNO together with a PACE-identical Lie estimator under the same equivariance and anchor supervision and the same parameter budget, but the estimator's outputs are never used for pullback or pushforward, so the prediction path is the plain FNO. It reports 0.66021±0.036440.66021 \pm 0.03644 on standard-resolution 2-D Burgers OOD, at the FNO level (0.63950±0.035590.63950 \pm 0.03559), while PACE-FNO reaches 0.05863±0.002870.05863 \pm 0.00287. The ordering is unchanged under α=0.10\alpha=0.10 observation noise (0.660160.66016 versus 0.058670.05867), under 5050\\% random sparse observations (0.66220.6622 versus 0.23970.2397), and under structured missingness (0.70160.7016 versus 0.32440.3244, with PACE-FNO below FNO+Aug at 0.35600.3560) (lines 441–446). Auxiliary parameters and supervision alone therefore do not close the gap; it closes only when the coordinate is applied through the canonical actions.

W2. Which components produce the advance

Appendix Table 7 (lines 1678–1695) now gives the component-wise decomposition. Disabling the learned pullback and pushforward at inference collapses OOD accuracy on every benchmark (1.03841.0384 on 1-D Burgers, 1.00851.0085 on 2-D Burgers, 0.72320.7232 on SWE, 0.62800.6280 on NS), and the matched control reproduces the collapse at the FNO level while keeping the same estimator, supervision, and parameter count (0.66021±0.036440.66021 \pm 0.03644 versus 0.05863±0.002870.05863 \pm 0.00287 on 2-D Burgers). The prescribed field actions carry the main improvement.

Within the canonicalizer, the analytic kinematic reduction is what stabilizes the tasks with background flow: a plain CNN that also regresses the background component degrades 2-D Burgers to 0.5478±0.03030.5478 \pm 0.0303 and is numerically unstable on SWE, whereas a plain CNN spatial branch that keeps the analytic background-flow extraction performs comparably to the full model on 2-D Burgers and SWE (0.0666±0.00760.0666 \pm 0.0076 and 0.1008±0.01420.1008 \pm 0.0142). On approximate NS rotations, removing the full Lie generator also removes the prescribed rotation–translation actions, so the canonicalization path becomes an identity map and the variant reduces to the plain FNO: its error (0.62800.6280) coincides with the without-canonicalization row (Appendix Table 7, lines 432–437 and 1678–1695). This row therefore measures the loss of the actions on the rotation family, not the Lie parameterization alone; evidence on the estimator structure comes from the 2-D Burgers and SWE rows above. The revised manuscript states these task-dependent differences explicitly in Section 5.3.

The advance therefore comes from the combination that the review identifies: amortized one-shot estimation supplies the spatial coordinate at inference, the analytic kinematic reduction removes a component that a learned regressor does not stabilize, and the prescribed spectral actions implement the exact transformation family (Appendix Table 7, lines 1678–1695; Appendix Table 16, lines 2160–2168).

Part 2/2 — Continued from Part 1/2 (W1 and W2)

W3. Isolating the contribution relative to amortized canonicalization

We agree that comparing one-shot prediction with per-instance optimization measures inference cost rather than isolating amortization. The revision separates the two comparisons. Against a straightforward learned canonicalizer, the plain-CNN variants in Appendix Table 7 (with the analytic background-flow extraction kept or removed) provide the matched comparison and locate the gain in the analytic kinematic reduction and the prescribed actions (see W2 above; Appendix Table 7, lines 1678–1695). Against iterative canonicalization, the dedicated same-split LieLAC+FNO run on 1-D Burgers gives 0.0342±0.01190.0342 \pm 0.0119 OOD error with an evaluation time of 8.738.73 s, versus 0.0267±0.00140.0267 \pm 0.0014 at 0.530.53 s for one-shot PACE-FNO (lines 415–418; Appendix Table 10, lines 2075–2083): on this split, amortized one-shot canonicalization matches per-instance optimization in accuracy within the seed spread of LieLAC+FNO, with smaller seed variance and about 16 times lower evaluation time. For the remaining settings, LieLAC's original evaluation uses different operators and task configurations, so we report those results as context rather than as matched comparisons (see also our response to Reviewer AYY8, W3; Appendix B.8, lines 1708–1711).

Dear Reviewer

We have only one day left for author-reviewer discussion. If the rebuttal clarifies your concerns, can you please acknowledge and increase the score.

Thanks

Reviewer_3ijnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Neural operator are sensitive to the observed coordinate frame : if a NO is trained on some given coordinate frame it will not give the same performance on a known dynamic in this "canonical" observation frame and a rotated / shifted observation frame. Authors propose to train a network that transform the input to project it to the canonical frame before giving this input to the Neural Operator which is doing the prediction in this canonical frame. They show it improves the results compared to training the FNO with the data augmentation.

Critical strengths
  • The method introduced by the authors leverage Lie theory for a physically consistent transformation of the input, which is interesting in the frame of PDE simulation as it restrict the type of transformations the encoder can apply.

  • They show improved results w.r.t data augmentation on several benchmark.

Critical weaknesses

Major :

(Disclosure : I am not very familiar with the extent of Lie Theory so that comment can be taken as a need for clarification rather than a stron critic.)

From my standpoint, the overall approach of the paper is very similar to Spatial Transformer Network (https://arxiv.org/abs/1506.02025), a classical paper in the CV community where you also learn a spatial transformation (translation/rotation / shear) on an input to facilitate a downstream task. It seems like most of the transformation this paper propose are basically restricted versions of this transformation, and that everything is trained in a similar way to the original paper. I know the STN network had a lot of ramifications, so I would not be surprised to see a temporal-based approach (where the transformation evolves smoothly with time) or a frequency-based approach where the transformation is done in Fourier space.

Globally it seems odd to me to not even cite the paper/ adress in a parallel the similarity with it, I think the Lie theory / authors applications could be interesting and justifiable from several reasons but today the paper is lacking a positioning w.r.t to a whole side of the community. (Globally, the comparison of data-augmented network versus coordinate transformer + network has been made already)

iIfthis is addressed in a proper manner, I would be happy to re-evaluate the paper.

Minor

  • Figures (3 /4/6) are incredibly small; it's very hard to read, especially Fig. 6. a
  • Some text (especially in the results section) is clearly written with an LLM and super hard to read to. For instance L465 "One-shot PACE-FNO reaches 89.953% on affNIST and 97.180% on homNIST, while LieLAC reaches 95.503% and 98.433% at much higher latency. Complete results are in Tables 16 and 17." I am not sure a 3 digit precision on the percentage make sense here, especially in the text.
Questions for authors
  • How does the paper position with respect to the STN community ? (And by that I don't mean running STN on the same example but rather on the conceptual side, is it a special case of STN, does similar work have been done on video for instance)
2 comments

Major: Relation to Spatial Transformer Networks

Thank you, Reviewer 3ijn, for pointing out the connection to Spatial Transformer Networks (STN). The analogy is apt, and the original manuscript should have cited this line of work. The revision now cites STN together with its recurrent and spatio-temporal extensions in Related Work (lines 110–119). Both families first estimate a transformation from the input and then pass the aligned input to a downstream predictor. For digit classification there is no time evolution: the terminal map ΓB\Gamma_B is the identity, and the input-canonicalization branch reduces to a restricted STN form (lines 115–117).

The PDE settings keep this input-alignment step and add the two parts that the dynamics provides: the field actions prescribed by each benchmark and the input-to-terminal-coordinate map ΓB\Gamma_B. In the Galilean case, the spatial position and the background velocity jointly determine the translation at the terminal time. Figure 3 shows, for a single prediction, the learned quantities, the analytic quantities, and the prescribed actions (lines 188–205); the benchmark-specific relations are given in Appendix A, Table 3 (lines 773–795). The alignment step is shared with STN. What the PDE setting adds is these dynamics-prescribed input and terminal actions.

The revision tests whether the gain comes from these actions with a matched control on 2-D Burgers: the same estimator, supervision, and parameter count, with only the input pullback and the terminal pushforward removed. The standard OOD error increases from 0.05863±0.002870.05863 \pm 0.00287 for PACE-FNO to 0.66021±0.036440.66021 \pm 0.03644 (lines 429–431). The prescribed field actions thus account for most of the improvement in this setting.

Q1: Conceptual positioning with respect to STN, and video tasks

The revision positions STN as the closest architectural relative of the input-canonicalization step. The digit settings realize a restricted STN form; the PDE settings additionally involve equation-prescribed field actions and the dynamics-determined input-to-terminal transformation (lines 110–119).

Video-oriented extensions are closely related. Sønderby et al. (2015) place a spatial-transformer module inside a recurrent network for cluttered-MNIST sequences, and Kim et al. (2018) estimate spatio-temporal transformations from neighboring frames to align video frames for restoration tasks. Both are now cited in Related Work (lines 110–114). In PACE-FNO, the terminal map is given by the PDE evolution relation, and periodic translations are implemented through the Fourier shift theorem. The revision lists video tasks with prespecified inter-frame dynamics as future work and adds no new video experiments (lines 118–119).

Minor: Figure readability

Figure 3 is now a formula-driven two-row flowchart with enlarged formulas and node labels (lines 188–201). Main-text Figure 4 keeps the larger 2-D Burgers result, and the full SWE terminal fields have moved to Appendix Figure 13 (lines 406–409; appendix lines 1483–1485). The former 1-D Burgers Figure 6(a) is split into a standalone full-width PACE-FNO figure, with the comparison methods shown separately as Figure 7 (appendix lines 1116–1117 and 1177–1178).

Minor: Result paragraphs and numerical precision

The affNIST and homNIST accuracies in the main text are now reported to one decimal place, with the inference times given at a readable precision; the complete results remain in the tables (lines 497–500). The revised sentences read:

On affNIST, one-shot PACE-FNO reaches 90.0% accuracy at 0.20 ms per sample, compared with 95.5% at 289 ms for LieLAC. On homNIST, it reaches 97.2% at 0.20 ms, compared with 98.4% at 288 ms for LieLAC.

The results paragraphs in Section 5 have also been rewritten for flow.

Thank you for your response, additional results and acknowledgment of the proximity with STN.

I think, given the proximity of this work with the STN literature that the video-oriented extensions needs to be addressed and compare with the method to publish in a ML-oriented conference. As such, it's hard to say if the presented methods will perform better than those temporal-oriented methods. I thus cannot recommend the acceptance of this paper.

I therefore maintain my score.

Reviewer_AYY8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes PACE-FNO, a wrapper for neural operators under prescribed geometric transformations. It estimates an input-frame coordinate, transforms the input toward a canonical frame, applies an FNO there, and restores the prediction using a transformation determined by the assumed dynamics. Experiments consider synthetic PDE settings with translations, Galilean shifts, and other transformations.

Critical strengths

Relevant problem formulation. The paper highlights an important source of distribution shift for neural operators: familiar dynamics observed in a changed coordinate or inertial frame. Separating frame alignment from canonical-frame dynamics prediction is an intuitive and potentially useful perspective, especially when the terminal frame differs from the input frame because of known dynamics.

Experiments show strong performance improvements with synthetically generated transformations. On the synthetic periodic translation and Galilean-shift PDE benchmarks, PACE-FNO substantially improves OOD prediction error over the reported FNO and data-augmented FNO baselines. These controlled settings directly test the paper’s proposed mechanism (canonicalising a transformed observation before prediction) and provide meaningful evidence that the approach can be effective when the transformation family and its dynamical effect are known.

Critical weaknesses

My main concern is that the presentation makes the method much harder to understand and assess than necessary. Some of the paragraphs are also composed of somewhat disconnected sentences which do not flow together. As a result, they can take a very significant effort to be read and understood.

Despite prior familiarity with neural operators, I could not reliably reconstruct the end-to-end computation or distinguish learned from analytic components without piecing together the main text and appendices. The paper also unfortunately lacks background on Lie algebras and related topics, and some related notation is introduced without prior definition (e.g., exponential maps on groups).

The comparisons to FNO and FNO+Aug are encouraging. However, because the central contribution is a canonicalisation-based approach to known geometric transformations, I would like to see more systematic comparison with the most closely related alternatives: LieLAC-style Lie-algebra canonicalisation and group-equivariant neural operators such as GFNO. The paper includes limited results in these directions, but a more comprehensive matched evaluation across the principal PDE settings would better establish when amortised canonicalisation provides an advantage over iterative canonicalisation or architectural equivariance.

I found the augmentation/training description somewhat ambiguous. Appendix A first describes constructing transformed inputs and corresponding terminally transformed targets, which suggests augmented supervised training pairs. In contrast, Algorithm 1 and the detailed-loss description state that the PDE loss is evaluated on clean pairs and that transformed inputs are used independently for coordinate-equivariance supervision. These may be compatible, but the manuscript should state unambiguously whether transformed input-target pairs ever enter LPDEL_{PDE}​, and provide the exact data flow for each loss.

Text font sizes in figures are quite small and often unreadable.

Questions for authors

It is unclear to me whether transformed input-target pairs are used in the supervised PDE loss during training, or whether transformed inputs are used only for the estimator equivariance loss. Could the authors clarify the data flow for LPDEL_{PDE}​, LequivL_{equiv}​, and LanchorL_{anchor}​, ideally with the clean, training-transformation, and OOD-transformation distributions for each main benchmark?

Could the authors state more explicitly what ΓB\Gamma_B​ is for each primary PDE benchmark? In particular, for Burgers and shallow water, how do the learned spatial coordinate and the analytically inferred background velocity combine to produce the output-time transformation?

The method seems to require that the transformation family, its action on fields, and its evolution from input to terminal time are known. Could the authors state clearly which of these are assumed given, which are learned, and whether the framework is intended to apply beyond prescribed symmetry-like transformations?

4 comments

Part 1/3 — W1–W3

Thank you, Reviewer AYY8, for identifying two parts of the method that were difficult to follow: readers had to piece together a single prediction from the main text and appendix, and the Lie notation lacked context. We have added a formula-based flowchart and a short introduction to Lie theory in Section 3. We have also clarified the training data flow and transformation ranges for each benchmark, and we now report GFNO on all four PDE benchmarks.

W1. Presentation and paragraph flow

We have reorganized Section 3 around a single prediction: the section first gives the full computation, then explains coordinate estimation, canonical-frame prediction, and terminal-frame restoration. Figure 3 is on lines 188–201. Background velocity and coordinate estimation are described on lines 218–234, and canonicalization and terminal restoration on lines 238–251. The main text now points to Appendix A for benchmark-specific settings (line 215; Appendix A, Table 3, lines 773–795; Table 4, lines 810–827).

W2. End-to-end computation and Lie algebra notation

Figure 3 (lines 188–201) places the learned and analytic parts of one prediction in the same diagram. The coordinate estimator predicts the spatial coordinate, and the known map ΓB\Gamma_B propagates it to the terminal time. The FNO predicts in the canonical frame, after which the terminal field action restores the requested output frame. The coordinate estimator and FNO are learned; the remaining quantities follow the specified physical relations for each benchmark (lines 203–205 and 215; Appendix A, Table 3, lines 773–795).

Lines 149–154 define the transformation group, Lie algebra, generator basis, exponential map, and field action. A translation example explains the meaning of exp⁡(ξ)\exp(\xi). The equations for background velocity and coordinate estimation follow on lines 218–225 and 227–234.

W3. Comparisons with GFNO and LieLAC

We report GFNO results on all four PDE benchmarks (Appendix Tables 10–13, lines 2071–2112). The PDE comparison with LieLAC is on 1-D Burgers; we also report results on two image benchmarks.

GFNO. GFNO and PACE-FNO use the same data splits and three random seeds. Their OOD errors and evaluation times are:

BenchmarkGFNO OOD errorGFNO time (s)PACE-FNO OOD errorPACE-FNO time (s)
1-D Burgers1.0244±0.00271.0244 \pm 0.00271.01480.0267±0.00140.0267 \pm 0.00140.5287
2-D Burgers1.0131±0.01191.0131 \pm 0.01190.74620.0755±0.01390.0755 \pm 0.01390.1399
2-D shallow water (SWE)0.7593±0.00710.7593 \pm 0.00710.71610.1069±0.01680.1069 \pm 0.01680.2952
Navier–Stokes0.4091±0.01180.4091 \pm 0.01189.29560.2766±0.03530.2766 \pm 0.03533.3728

PACE-FNO has the lower OOD error on all four benchmarks. The ratio ranges from about 1.5 on Navier–Stokes (0.40910.4091 versus 0.27660.2766) to about 38 on 1-D Burgers (1.02441.0244 versus 0.02670.0267). The transformation families explain the gap. For 2-D problems on square grids, GFNO uses the discrete p4p4 group, which covers integer translations and 90∘90^\circ rotations. Our OOD transformations also include subpixel translations, Galilean boosts, and arbitrary-angle rotations for Navier–Stokes. For SWE, GFNO uses weight decay of 10−310^{-3}, while PACE-FNO and FNO+Aug use 10−410^{-4}. The revised table caption records this difference. These results indicate where canonicalization helps: when the test transformations are continuous or act through the dynamics, as Galilean boosts do, and so lie outside the discrete group built into GFNO. All rows use the cross-resolution OOD protocol (642!→!128264^2\\!\to\\!128^2 for the 2-D benchmarks). The standard-resolution 2-D Burgers OOD errors quoted in the responses to Reviewers 3ijn and QYkT (e.g., 0.058630.05863) come from a separate diagnostic split at the training resolution.

LieLAC. On the same 1-D Burgers split, LieLAC+FNO has an OOD error of 0.0342±0.01190.0342 \pm 0.0119 and a recorded evaluation time of 8.73 s. One-shot PACE-FNO has an error of 0.0267±0.00140.0267 \pm 0.0014 and an evaluation time of 0.53 s (main text lines 415–418; Appendix Table 10, lines 2075–2083). The two mean errors differ by less than the seed spread of LieLAC+FNO. We therefore claim comparable accuracy with smaller seed variance and about 16 times lower evaluation time, and we do not claim higher accuracy.

On affNIST and homNIST, LieLAC is more accurate, while PACE-FNO has lower per-sample inference latency. On affNIST, PACE-FNO reaches 89.95389.953\\% at 0.202 ms per sample, compared with 95.50395.503\\% at 288.592 ms for LieLAC. On homNIST, the results are 97.18097.180\\% at 0.200 ms for PACE-FNO and 98.43398.433\\% at 287.818 ms for LieLAC. PACE-FNO estimates coordinates in one pass at inference, whereas LieLAC optimizes them separately for each test sample. This difference matters when evaluating many samples (Appendix Table 18, lines 2220–2234).

Part 2/3 — Continued from Part 1/3 (W3–W5 and Q1)

The authors’ public LieLAC implementation provides runnable code for 1-D Burgers and image tasks. Other PDE results in the original paper use different operators and task settings. The 1-D Burgers experiment therefore provides our direct PDE comparison, while the other reported results give context for LieLAC’s broader evaluation. We do not claim a matched LieLAC comparison on the 2-D PDE benchmarks.

W4. Ambiguity in the augmentation and training description

Both descriptions were correct, but they applied to different benchmarks and the original text did not say so. The 1-D Burgers PDE loss uses transformed supervised pairs, while the other PDE benchmarks use clean supervised pairs. The revision states the data flow of each loss per benchmark (main text lines 263–280; Appendix A, lines 801–852).

BenchmarkPDE lossEquivariance termAnchor term
1-D BurgersDynamics-consistent transformed pairs (x+,y+)(x^+,y^+)Compares estimator responses to clean and transformed inputsNone
2-D BurgersClean pairs (x,y)(x,y)Transformed input x+x^+Clean input
SWEClean pairs; also includes an invariant-height lossTransformed inputClean input
Navier–StokesClean pairs (x,y)(x,y)Transformed inputClean input

For SWE, the invariant-height term compares the clean and transformed inputs after each has been canonicalized.

W5. Figure readability

We have revised the figure layouts. The PDE examples are now stacked vertically, the five-panel Navier–Stokes figure is arranged as three panels above two, and the three WeatherBench2 panels have been moved to the appendix and arranged one per row.

Q1. Loss data flow and distributions of clean, training-transformed, and OOD-transformed data

For 1-D Burgers, LPDEL_{\mathrm{PDE}} is evaluated on dynamics-consistent transformed pairs (x+,y+)(x^+,y^+). For the other three PDE benchmarks, it is evaluated on clean pairs (x,y)(x,y). In every benchmark, LequivL_{\mathrm{equiv}} uses a clean input xx and its training-transformed version x+=Cexp⁡(Δ0)xx^+=\mathcal C_{\exp(\Delta_0)}x, and requires the group-coordinate difference between the estimator outputs to match the known perturbation Δ0\Delta_0. Except for 1-D Burgers, LanchorL_{\mathrm{anchor}} is evaluated on clean inputs and penalizes spatial coordinates that deviate from the canonical slice. SWE also includes an invariant-height loss, which compares the height fields obtained by canonicalizing the clean and transformed inputs. The table below gives the clean data source and the transformations used during training and OOD evaluation (main text lines 263–280; Appendix A, loss definitions, lines 801–852; clean data and transformation supports, lines 797–827).

BenchmarkClean data sourceTraining transformationsOOD transformations
1-D BurgersInitial conditions drawn from a periodic Gaussian random fieldδx∼U[−0.1,0.1]\delta_x \sim U[-0.1,0.1], vbg∼U[−0.2,0.2]v_{\mathrm{bg}} \sim U[-0.2,0.2]200 samples at resolution 2048; translation U[0.2,0.8]U[0.2,0.8], Galilean shift c∼U[−1,1]c \sim U[-1,1]
2-D BurgersCanonically centered Burgers states(tx,ty)∼U[−0.1,0.1]2(t_x,t_y) \sim U[-0.1,0.1]^2, Ubg∼U[−0.5,0.5]U_{\mathrm{bg}} \sim U[-0.5,0.5]200 samples at resolution 1282128^2; (tx,ty)∼U[−0.3,0.3]2(t_x,t_y) \sim U[-0.3,0.3]^2, Ubg∼U[−2,2]U_{\mathrm{bg}} \sim U[-2,2]
SWECanonically centered states with zero background velocityΔx∼U[−0.15,0.15]2\Delta\mathbf{x} \sim U[-0.15,0.15]^2; Fr∼U[0,0.2]\mathrm{Fr} \sim U[0,0.2], with direction uniform on [0,2π][0,2\pi]200 samples at resolution 1282128^2; Fr∼U[0.3,0.8]\mathrm{Fr} \sim U[0.3,0.8], wave center uniform on the periodic grid, and amplitude log-uniform on [0.5,2.5][0.5,2.5]
Navier–StokesHead-on dipole-collision trajectoriesθ∼U[−π/4,π/4]\theta \sim U[-\pi/4,\pi/4], (tx,ty)∼U[−0.05,0.05]2(t_x,t_y) \sim U[-0.05,0.05]^2200 samples at resolution 1282128^2; θ∼U[−π/2,π/2]\theta \sim U[-\pi/2,\pi/2], (tx,ty)∼U[−0.1,0.1]2(t_x,t_y) \sim U[-0.1,0.1]^2

Each OOD set was generated independently and excluded from training and model selection.

Part 3/3 — Continued from Part 2/3 (Q2 and Q3)

Q2. The benchmark-specific ΓB\Gamma_B

The map ΓB\Gamma_B takes the input coordinate to the terminal coordinate. The estimator learns the spatial coordinate, while background velocity is computed analytically from the observed field (Appendix A, Table 3, lines 773–795).

BenchmarkTerminal coordinateLearned quantityAnalytic quantity
1-D Burgersδ^x,T=δ^x+v^bgT\hat\delta_{x,T} = \hat\delta_x + \hat v_{\mathrm{bg}}TSpatial phase δ^x\hat\delta_xMean background velocity v^bg\hat v_{\mathrm{bg}}
2-D Burgerst^T=t^+(U^bgT,,U^bgT)\hat{\mathbf t}_T = \hat{\mathbf t} + (\hat U_{\mathrm{bg}}T,\\, \hat U_{\mathrm{bg}}T)Spatial translation t^\hat{\mathbf t}Scalar mean background velocity U^bg\hat U_{\mathrm{bg}}
SWEΔx^T=Δx^+V^bgT\hat{\Delta\mathbf{x}}_T = \hat{\Delta\mathbf{x}} + \hat{\mathbf V}_{\mathrm{bg}}TSpatial translation Δx^\hat{\Delta\mathbf{x}}Mean velocity vector V^bg\hat{\mathbf V}_{\mathrm{bg}}
Navier–Stokesξ^T=ξ^0\hat\xi_T = \hat\xi_0Rotation and translationNo additional kinematic drift

Q3. Given information, learned quantities, and intended scope

For each task, the transformation family, its field action, the generator basis, the terminal map ΓB\Gamma_B, and the sampling range are specified. The network learns the spatial-coordinate estimator and FNO. For PDEs with a kinematic component, the background velocity is computed from the observed field. Test coordinates may lie outside the training range, provided the transformation family and its field action are known and computable (main text lines 203–205; Appendix A, Table 3, lines 773–795). The framework is intended for prescribed transformation families. It does not discover an unknown symmetry from data.

The WeatherBench2 experiment probes this boundary. It evaluates natural recent-time distribution shift and extreme-heat events. We used periodic longitude shifts: during training, inputs, static fields, and targets were shifted together; PACE estimates a longitude displacement from context and applies a bounded correction to the FNO output. We did not use rotations or latitude shifts. At this data scale, implementing full field canonicalization exceeded the GPU memory and system RAM available to us at the time, so we evaluated only the longitude-shift correction. PACE-FNO and FNO performed similarly in this constrained setting. The experiment therefore does not determine whether full canonicalization would benefit WeatherBench2 (main text lines 502–512; Appendix B.12, lines 2054–2061).

Thanks for addressing most of my concerns. The paper is looking better now. I've updated my score.

A few minor issues:

  • ΓB\Gamma_B is now mentioned in the related work discussion in Sec. 2.3 without prior introduction. If you need to refer to explicit architectural components, it might be easier to move the related work section to somewhere later in the paper, somewhere after the methodology section.
  • In Sec. 3.4, it would be better to name and briefly explain the three loss components in the main text, even if you can't fit their specific mathematical details there. Otherwise, the formula has little meaning. You're already naming them as "the equivariance term" and "the anchor term". You could at least add the symbols next to the names, i.e., "The equivariance term LequivL_{\mathrm{equiv}} compares...". That'd be the most compact way of addressing this.
  • Please clarify in the WeatherBench2 section that the evaluated variant uses a longitude-based output correction, rather than the full input-canonicalization pipeline. This would help readers interpret the comparison with FNO.

Comments on the paper

1 comment

Dear authors and reviewers,

The author-reviewer discussion window will close tomorrow on November 18.

@authors: If not done already, please respond to the reviewers' comments before the discussion window closes. I would advise short and concise responses over lengthy explanations.

@reviewers: Please check the authors' responses and provide any additional comments before the discussion window closes. Consider updating your initial review if necessary, and in particular your score if your opinion has changed. You may increase your score if you feel the authors' responses warrant it, maintain your score if your opinion remains the same, or even decrease your score if you feel the other reviews or the authors' responses reveal new shortcomings. At the very least, please acknowledge the authors' responses.

Thank you for your attention and cooperation.

The AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.