acceptodds
Under review as a conference paper at ICLR 2027

One-Way Coupling via Patchwise Adaptive Normalization in Transformer Surrogates

Abstract

Many physical systems exhibit one-way coupling: exogenous fields such as medium density or obstacle geometry influence the dynamics without being affected by them. We investigate how explicitly representing these fields can improve the accuracy and computational efficiency of transformer-based PDE surrogates. Building on spatially adaptive normalization, we introduce Patchwise Adaptive Normalization (PAdaNorm), which partitions exogenous fields into patches aligned with the state tokenization. We evaluate PAdaNorm on datasets from three different domains, using a Walrus-inspired transformer architecture. Across the three PDE families, PAdaNorm is on par with or up to 16% better than parameter-matched baselines in mean long-horizon variance-scaled RMSE on 60-step autoregressive rollouts. We also find that PAdaNorm better maintains hard physical constraints over long rollouts, such as preventing flow into impenetrable solid regions in porous media. When the conditioning is static, PAdaNorm caches the conditioning embeddings once and reuses them across the entire rollout, reducing inference FLOPs per step by up to 36% compared with parameter-matched baselines. An ablation over eight conditioning variants shows that modulating a single attention axis (temporal or spatial) suffices and that a lightweight shared MLP across blocks remains competitive with per-block projections.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Partial differential equations, PDE surrogates, One-way coupling, Physics foundation models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EbWvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to condition transformer-based PDE surrogates on static, spatially varying inputs such as material density, speed of sound, obstacle masks and pore masks. These inputs influence the dynamics but are not changed by them, and the paper calls them one-way coupled or exogenous fields. The proposed method, Patchwise Adaptive Normalization (PAdaNorm), partitions the conditioning fields into the same PxP patches used to tokenize the state. A shared MLP maps each patch to a conditioning vector, and a per-block linear projection turns that vector into per-token scale, shift and gate parameters in the adaLN-Zero style.

Critical strengths

s1. The question this research follows is practical with clear motivation.

s2. The paper itself is clear and easy to follow.

s3. The evaluations are done over three seeds raising confidence on the reported results.

Critical weaknesses

(w1) It is unclear how the novelty of this work goes beyond SPADE, and FlowAR's spatial AdaLN. Although, the adaptation to PDE surrogate seems interesting the methodological novelty of the framework itself is limited.

(w2) Given w1 the finding can still be made more interesting by applying the same technique across a wider range of benchmarks and providing a comprehensive evaluation of this approach on PDE benchmarks and under different methods of solution generation.

(w3) The global-AdaNorm baseline average-pools the conditioning fields to C_cond scalars before the learnable encoder, so on acoustics the model has almost no access to the scattering geometry. This can explain most of the improvement rather than the spatial conditioning. Other works exist without the pooling and with a learnable layer that directly maps the exogenous information to scale and shift without any mean pooling. These are still variations of global-AdaNorm which may help (see SEA [1]).

(w4) GNOT's results suggest that there maybe implementation issues. The results for GNOT is an order of magnitude above every other benchline. This needs clarification, and if the implementation matches the exact implementation bencline then a description to why the authors suspect this is the case.

(w5) The reproducibility does not require the full training infrastructure. The implementation can be simply provided through an algorithm within the paper or even a psudocode in the appendix so it is possible to evaluate the exact conditioning mechanism that is proposed in this work.

[1] Esmati, P., Dadashzadeh, A., Ardakani, V.G., Larrosa, N. and Grilli, N., 2024. SEA: State-Exchange Attention for High-Fidelity Physics Based Transformers. Advances in Neural Information Processing Systems, 37, pp.47317-47343.

Questions for authors

Q1. Can the authors evaluate on additional datasets with static exogenous fields, ideally including a third PDE family beyond acoustics/Navier–Stokes/porous media and at least one of the motivating cases in the introduction (thermal stress, external potential, refractive index), none of which is tested?

Q2. Can the authors add a global-AdaLN control that does not average-pool the raw fields (See w3). A recommendation would be the same per-patch encoder f_θ followed by attention-pooling of the patch embeddings and, separately, in-context concatenation combined with global AdaLN?

Q3. Given the code cannot be released can the authors provide pseudocode for the full PAdaNorm block with explicit tensor shapes at every step? An algorithm or a pseudocode independent of the proprietary training infrastructure will still be needed to reproduce, evaluate and adapt the proposed method.

I am willing to reconsider my score if these questions are addressed with less emphasis on the right away benchmarks since I understand the limited period of rebuttal.

3 comments

We thank reviewer EbWv for their careful read of our paper and the insightful comments and questions. Let us address each one by one.

Weaknesses

Weakness 1: While we acknowledge that novelty is somewhat a subjective concept, we would like to challenge this statement from Reviewer EbWv. One can always describe ViT are merely an application of transformers to the image domain or Diffusion Models can be considered as applying diffusion methods for image generation, but this would not capture the rationale and work behind the proposed approach.

Weakness 2: We fully agree that increasing the diversity of application domains on which to evaluate PadaNorm would increase the confidence in the benefits of the method. The lack of additional domains in the paper is limited by the scarce availability of open-source dataset that satisfy (i) one-way coupled spatially varying fields (ii) long enough rollouts. We searched for openly available datasets satisfying both requirements and, to the best of our knowledge, the datasets used in the paper are the only ones currently available.

Weakness 3: See our answer to Question 2. We thank the reviewer for the suggestion and have added those experiments in the manuscript.

Weakness 4: We have removed GNOT from the revised manuscript as our adaptation of the original approach to this setup might not be suitable.

Questions

Question 1: As discussed in our answer to Weakness 2, to the best of our knowledge no other open dataset currently satisfies both requirements. We agree that extending the evaluation to further PDE families, such as the motivating cases of the introduction, is an important direction for future work. Would Reviewer EbWv have any dataset suggestion that we would have missed on which we could run some additional experiments?

Question 2: We have now run the experiments and added both suggested controls to Table 1, on all six settings: attention-pooled global AdaLN, and attention-pooled global AdaLN combined with in-context concatenation. Attention pooling alone performs on par with average-pooled AdaNorm (average vRMSE 0.680 vs. 0.665) and remains well behind PAdaNorm on the acoustic datasets. Combined with in-context concatenation, it is a strong alternative: it attains the lowest error on Maze and Joint, and a slightly lower average vRMSE than PAdaNorm (0.480 vs. 0.484). PAdaNorm keeps the best average rank (2.17 vs. 2.67) and the lowest error on Inclusions, Discontinuous and FlowBench. We thank again the reviewer for this suggestion.

Question 3: This is indeed a valid request, and we have updated the manuscript with a pseudocode implementation.

I would like to thank the authors for their response.

Novelty and impact: An argument that compares methodologies between this work and other advances in the field irrelevant to this work (ie. ViT, or diffusion) is not helpful for the process of this rebuttal. My weakness 1 was specifically regarding the novelty compared to the most related works here (I specifically named your related works SPADE and FlowAR). Although pure methodological novelty is not a point to alter score on (as pointed in the original review), it is nevertheless important to understand the impact of the work which is something that alters the score.

Remaining issue: Proposed contribution PAdanorm comes with a core claim regarding achieving improved or similar results at fewer FLOPs. However the following issue remain:

  1. The impact in its current form is minimal. Regardless of the novelty, the presented results in the main table (Table 1) are all within the noise range across measured seeds with in-context (wide), and the newly added AdaNorm (attn-pool + in-context).

  2. The computation improvements (in terms of forward pass FLOPs) is also small compared to in-context (wide), and is in fact worse than the newly added AdaNorm (attn-pool + in-context).

Consequently after carefully reading the rebuttals and authors response to other reviewers I stand by my original score.

AuthorsOpenReview ↗

Response to Reviewer EbWv

We thank Reviewer EbWv for the follow-up.

We apologize if the comparison to ViT and diffusion models read as a deflection as that was not our intent, and we agree it was an unhelpful analogy. The point we wanted to make concerns SPADE and FlowAR specifically: spatially adaptive modulation is a known primitive, and our contribution is not the primitive itself but its adaptation to autoregressive PDE surrogates with one-way coupled fields.

A key difference between PadaNorm and SPADE and FlowAR is that in both, the conditioning is part of the sample being generated. In PDE surrogates, the conditioning fields (density, sound speed, obstacle/pore masks) are exogenous: they drive the dynamics but are never changed by them, and they are static over the whole rollout. This is what makes one-way coupling exploitable: PAdaNorm computes the conditioning embeddings and per-block modulation parameters once per rollout and caches them, which has no analog in image generation.

On "results within noise" (Table 1). We thank the reviewer for raising this point.

First, for patchwise vs. global conditioning, the two purely global controls isolate the question the reviewer raises, and here the advantage of patchwise conditioning is consistent. PAdaNorm attains lower long-horizon error than both global AdaNorm and attention-pooled AdaNorm in 5 of 6 settings, with a median reduction of about 23%. The only exception is CO₂, where all methods lie within about 1% of each other. Across seed-matched runs, PAdaNorm is lower in 14/18 and 15/18 cases respectively (Wilcoxon signed-rank, p = 0.001 and p < 0.001).

The attention-pooled + in-context control is not purely global: concatenating the conditioning fields to the input gives it spatially resolved information. It is therefore a hybrid, and its competitiveness supports our central claim that spatially resolved conditioning is the key ingredient. Even so, PAdaNorm attains the best average rank of all eight methods (2.17 vs. 2.67). It is also best on more datasets than any other method: Inclusions, Discontinuous and FlowBench, against Maze and Joint for the control. The control's average vRMSE is lower by only 0.004 (0.480 vs. 0.484), and this comes from a single datasets of the ones that we tested (Maze). Excluding Maze, PAdaNorm's average error is lower by 0.019. A paired Wilcoxon signed-rank test finds no evidence of an advantage for the control (W = 10, p = 1.00 over the six settings; p = 0.93 over 18 seed-matched runs). PAdaNorm thus matches the strongest hybrid control method overall while being the more consistent method across heterogeneous PDE families.

Reviewer_9eayOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates how to include spatially varying conditioning fields into neural PDE solvers. More specifically, it proposes to use adaptive normalization (AdaLN) in the patching-based Walrus vision transformer. Instead of providing global conditions for all patches, the authors include a patch-wise conditioning (PAdaNorm). The experiments compare the proposed PAdaNorm to in-context conditioning, global AdaLN, and cross-attention. PAdaNorm especially lowers the rollout error on acoustic scattering tasks. On the FPO and CO2 task, the method performs similarly to global AdaLN.

Critical strengths
  • The experiments show that the approach delivers good results on acoustic scattering compared to other well-established conditioning methods
Critical weaknesses
  • The biggest downside is that the paper has limited novelty, as it is only proposing to use the AdaLN normalizer for each patch when moving to spatially varying conditioning fields.
  • shows an error advantage only for the acoustic scattering PDE family
Questions for authors
  • Have you tried PAdaNorm also for surrogates based on the vision transformer other than Walrus?
2 comments
AuthorsOpenReview ↗

Answer to Reviewer 9eay

We thank Reviewer 9eay for their review of our work.

Let us try to answer the few points raised in their review:

Reviewer 9eay mentions how our paper has limited novelty. While we acknowledge that novelty is somewhat a subjective concept, we would like to challenge this statement from Reviewer 9eay. One can always describe ViT are merely an application of transformers to the image domain or Diffusion Models can be considered as applying diffusion methods for image generation, but this would not capture the rationale and work behind the proposed approach.

Weakness 1: PadaNorm can indeed be described as “only” applying Adaptative Normalization differently to each patch. However, we argue that the proposed method enables a better conditioning of the state variables because it accounts for one-way coupling conditioning not being spatially equivariant. Hence, it improves long roll-outs both quantitatively (as demonstrated by the performance gains obtained by our method on FlowBench and acoustic datasets), and qualitatively as it better captures the underlying physics as shown on the CO2 dataset, and it requires less compute than In-context models with the same number of parameters (5–21% fewer FLOPs per forward pass during training, and 8–29% fewer FLOPs per rollout step at inference).

Weakness 2: Regarding the fact that PadaNorm “shows an error advantage only for the acoustic scattering PDE family”, as also discussed in our previous paragraph, we show that PadaNorm indeed improves performance on acoustic datasets, but also for Flow Simulation (FlowBench dataset). Regarding the CO2 dataset, our qualitative analysis shows that PadaNorm enables the model to better preserve and integrate the underlying physics. Most importantly, on all datasets, the error of PAdaNorm is lower than, or within seed variation of, the error of the In-context models with the same number of parameters, while PAdaNorm requires less compute for both training and inference. This is because PAdaNorm adds its parameters to the conditioning encoder, which runs once per rollout, whereas the larger In-context models add them to the backbone, which runs at every step.

Question 1: This is a very relevant question and we actually haven’t tried training a ViT in this setup. This is mainly because, while ViT might appear as a valuable choice as an architecture for this setup, it has been shown in previous work how vanilla ViT are not suited for Physics PDE [1], [2] and most state-of-the-art works have resorted to other architectures or modified the Transformer architecture [1], [2], [3], [4].

[1] Walrus: A Cross-Domain Foundation Model for Continuum Dynamics, McCabe et al., 2026

[2] ReViT: Rotational-equivariant Vision Transformers for Neural PDE Solvers, Wei et al., ICML 2026

[3] Poseidon: Efficient Foundation Models for PDEs, M. Herde et al., 2024.

[4] PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations, B. Holzschuh, 2025.

Thank you for your response. On FPO, while PaDaNorm is technically the best one, the difference between the baselines seems rather small, suggesting that the conditioning paradigm in general may not be the most relevant part here.
My main concern about the limited novelty also still stands, and hence I maintain my original rating.

Reviewer_kKB6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Patchwise Adaptive Normalization (PAdaNorm) which is a more expressive conditioning method than the commonly used adaptive instance normalization. Instead of using the same learnable scale and shift vectors for each patch, PAdaNorm modulates each patch individually based on a spatially specific conditioning embedding.
PAdaNorm is evaluated against a set of alternative conditioning mechanisms across several 2D datasets in long rollout scenarios. Additionally, the authors test different tokenization strategies for obtaining the conditioning embeddings. The findings indicate that PAdaNorm achieves better performance than alternative methods at a comparable parameter/FLOP count.

Critical strengths

PAdaNorm is a significantly better conditioning mechanism than adaptive instance normalization if the conditioning is a full field, since it does not compress the full field into a single conditioning vector, but instead allows a targeted patch-wise conditioning. This is very intuitive and also supported by the experimental evidence presented in this paper. Modulation via learnable scale and shift vectors is very common in transformer-based neural surrogates and I think this work explores an interesting and relevant question.

Critical weaknesses
  • The results could be stronger. PAdaNorm shows better performance than the alternative methods in the long-horizon experiment [Table 1] on the acoustic scattering datasets, but improvements for FlowBench FPO and the CO2-water flow dataset do not seem very high. For FlowBench FPO, the in-context (wide) setup seems to be within seed-noise of PAdaNorm and for CO2 water flow, the global AdaNorm is better, which is counterintuitive. If I understand correctly, PAdaNorm is better on rock pixels, but since all fields must be zero for those, this can be fixed in post-processing.

  • Also in Table 1, the VRMSE for the acoustic scattering datasets Maze, Inclusions and Discontinuous are much worse than the ones reported in [1] (Walrus, McCabe et al. 2026), for example Maze 0.1390 MPP-AViT-L vs. 0.34 PAdaNorm. The reason for this is most likely that the architectures used as baselines in [1] are much larger. However, this makes me a less confident in the training setup and whether models are fully converged or whether the results are representative of larger models. The main question for me is whether PAdaNorm is better than the in-context conditioning. A setup that is comparable to the one in [1] and where PAdaNorm beats the existing baselines would be much more reliable to answer this question.

  • The ablations [Table 2, Table 3] don't show a clear winning config. While this is an honest and interesting finding, it makes it difficult to draw conclusions which type of conditioning should be used. There is no clear analysis or insight depending on the type of problem which config should be used. This makes adoption of this conditioning mechanism more difficult in practice.

  • The implementation and idea is similar to SPADE (Park et al., 2019), FlowAR (Ren et al., 2025) and PixelDiT (Yu et al., 2026) as correctly mentioned by the authors. While the spatially adaptive modulation based on conditioning fields has never been used in that way by neural surrogates, as far as I know, this limits the technical novelty. PAdaNorm seems like an adoption of a common method in computer vision.

  • Some minor improvements in the presentation. For example, it was not immediately clear to me what the different conditionings in Table 2 mean and I had to check the descriptions in the appendix to fully understand it. Also, it would be better to list each dataset VRMSE of the joint model individually (Table 2).

Questions for authors
  • Do you think the improvements from PAdaNorm compared to in-context conditioning will hold in settings with larger architectures/improved training setups such as the ones in McCabe et al. (2026)?
  • What is your general recommendation what version (cf. Table 2 and Table 3) of PAdaNorm to use? Based on a specific dataset, are there any insights that could inform the choice?
  • In Table 3, you change the tokenization of the conditioning field. Have you also considered changing the tokenization of the other field inputs together with it?
  • The experiments are restricted to 2D PDEs. How would PAdaNorm work in settings where we are conditioning on general 3D geometries, for example the geometry branch in AB-UPT (Alkin et al., 2025)? Does it make sense to have a spatially adaptive modulation here and how could that be implemented?
2 comments
AuthorsOpenReview ↗

Answer to Reviewer kKB6 (1/2)

We thank Reviewer kKB6 for reviewing our work and highlighting some points that could be clarified and/or improved.

Let us try to address the main points raised in their review.

Weaknesses

Weakness 1: Reviewer kKB6 mentions how “results could be stronger” on some tested benchmarks. PAdaNorm achieves the best average rank across the six settings (2.17, next best 2.67). It attains a lower mean error than every In-context control, including the parameter-matched ones, on the four acoustic settings, and a lower mean error than the parameter-matched In-context (wide) control on all six settings (Wilcoxon signed-rank, p ≈ 0.03), although on FlowBench and CO2 the differences are within seed variation. On these two datasets, PAdaNorm however requires less compute than the In-context models with the same number of parameters (In-context wide and In-context deep): it uses 5–21% fewer FLOPs per forward pass during training and 8–23% fewer FLOPs per rollout step at inference. This is because PAdaNorm adds its parameters to the conditioning encoder, which runs once per rollout, whereas In-context wide and In-context deep add them to the backbone, which runs at every step. Regarding the CO2 dataset, we thoroughly discuss from lines 321 to 327 in the original manuscript how PadaNorm does not necessarily achieve better performance than alternatives in terms of vRMSE, but it better follows the physics at its errors “make more sense” than that of competing conditioning methods.

Weakness 2: As discussed in the manuscript, the present work does not aim at outperforming Physics Foundation Models with significantly more parameters, but rather demonstrating that for modelling Physics PDEs our proposed methods, PadaNorm, outperforms existing, parameter-matched, conditioning methods as it enables to better capture the underlying physics. To discard any confounding factors we use the same Walrus-like architecture with the same number of parameters (except for those related to the conditioning method itself) for all tested conditioning set-ups and only trained on one dataset at the same time (except for the Joint dataset). Finally, the vRMSE gap noted by the reviewer reflects scale rather than architecture issues. On Maze, MPP-AViT-L scores 0.1390 (median), while PadaNorm scores 0.344 (mean). MPP-AViT-L however is a 407M-parameter model pretrained on several 2D PDE datasets and then finetuned on 4.5M Maze samples; while PadaNorm in our test setup is a 19.1M-parameter model trained from scratch on 1.6M samples. Despite the 21× size gap, PAdaNorm already improves over MPP-AViT-L on Inclusions (0.426 vs. 0.494). Regarding convergence, we have added to the appendix the training curves of PAdaNorm spatial-only on each dataset, for the three seeds reported in Table 1.

Weakness 3: To facilitate selecting the best PadaNorm version we added the average vRMSE to Table 2. This demonstrate that the spatial PadaNorm conditioning is the best version. As for Table 3, we have added this ablation to show that the shown performance have not been “over”-optimized and that the performance demonstrated by PadaNorm are robust to different patch conditioning vector computation.

Weakness 4: PadaNorm could indeed be described as adapting some existing CV concepts to Physics PDE. While we acknowledge that novelty is somewhat a subjective concept, we would like to challenge this. One can always describe ViT are merely an application of transformers to the image domain or Diffusion Models can be considered as applying diffusion methods for image generation, but this would not capture the rationale and work behind the proposed approach.

Weakness 5: We would like to thank Reviewer kKB6 for pointing this out. We have included those modifications in the manuscript.

AuthorsOpenReview ↗

Answer to Reviewer kKB6 (2/2)

Questions

Question 1: We cannot answer this from our experiments: scaling studies such as McCabe et al. grow data and model size jointly, whereas all our models are trained on a single dataset with a fixed budget. It is possible that, at higher scale, in-context conditioning start outperforming PadaNorm. Two points nonetheless argue for the relevance of PAdaNorm. (i) The regime we study is common in practice. Physical data (simulated or real) is often expensive and scarce. Therefore, large-scale results might not always be the most relevant to practitioners in practice. (ii) For the same number of parameters, PAdaNorm is more accurate and requires less compute at inference. The In-context models with the same parameter count as PAdaNorm (In-context wide and In-context deep) have a higher mean error on all four acoustic settings, and they require 8–29% more FLOPs per rollout step. The difference comes from where the additional parameters are placed. In-context wide and In-context deep add them to the backbone, which runs at every rollout step. PAdaNorm adds them to the conditioning encoder, and since the conditioning fields do not change over time, the conditioning encoder runs only once per rollout.

Question 2: Our recommendation is to use the spatial-only variant with per-block projections by default: it attains the lowest average error across the four acoustic settings of Table 2 and is best on Inclusions and Discontinuous. When a single model is trained on several datasets, or parameters are constrained, the shared per-axis variant is a strong alternative: it is best on Joint and second on average (0.470), with 12% fewer parameters. We have added this recommendation to Sec. 5.1 of the revised manuscript. In all cases, and as expected, the conditioning should not be coarser than the token grid (Sec. 5.2, Table 3).

Question 3: We deliberately did not experiment with the tokenization of the input fields other than the conditioning fields, as this has been studied extensively in the literature and is not the question we aim to answer in this work, namely how best to encode conditioning fields. In future work, it would however be interesting to study the interaction between the encoding of the input fields and that of the conditioning fields.

Question 4: PadaNorm can easily be extended to more spatial dimensions or mesh geometries: it only requires that each token receive one conditioning vector computed from the conditioning fields around it. When the 3D conditioning is a volumetric field on a grid, we can simply partition the fields into P×P×P patches aligned with the state tokens, and f_θ maps each flattened patch (C_cond·P³ values) to its conditioning vector. For geometries given as surface point clouds, as in AB-UPT, we can replace the patch by a neighborhood: for each anchor token at position x_i, the conditioning vector can be obtained by pooling the geometry points within a radius of x_i. Each block then modulates that token with its own scale, shift and gate.

Comments on the paper

1 comment

Dear Reviewers, ACs and PCs,

We sincerely thank the reviewers for their time and thoughtful comments. We are encouraged that the reviewers found our research question practically motivated (Reviewer EbWv), our paper clear and easy to follow (Reviewers 9eay, EbWv), and our evaluation reliable with its three-seed protocol (Reviewer EbWv). We are also glad that the reviewers acknowledged the strong long-rollout performance of PAdaNorm on the acoustic scattering benchmarks (Reviewers kKB6, 9eay) and its parameter- and compute-matched advantage over in-context conditioning (Reviewer kKB6).

In response to the comments, we have carefully revised and strengthened the manuscript with the following discussions and analyses:

  • Per-dataset average VRMSE and clarified descriptions of the conditioning variants in Table 2, plus the joint model's per-dataset VRMSE in the appendix (Reviewer kKB6)
  • An explicit recommendation of which PAdaNorm variant to use, depending on the setting (Section 5.1) (Reviewer kKB6)
  • Pseudocode of the full PAdaNorm block with explicit tensor shapes at every step (Reviewer EbWv, Q3)
  • Training curves of PAdaNorm spatial-only on each dataset, in the appendix (Reviewer kKB6)
  • Removal of the GNOT baseline, whose implementation we found to be broken (Reviewer EbWv, W4)

Every addition has been added to the manuscript in blue to make them identifiable.

In addition, to address the reviewers' concerns, we provide the following new experiments and analyses in the individual responses:

  • Two attention-pooled global AdaLN controls that do not average-pool the raw conditioning fields (pooling by multi-head attention, Lee et al., 2019), one of them combined with in-context concatenation, run on all six settings: the combined variant is best on Maze and Joint, while PAdaNorm keeps the best average rank (Reviewer EbWv, W3/Q2)
  • Statistical significance of PAdaNorm vs. the parameter-matched In-context (wide) control across all six settings (Wilcoxon signed-rank, p ≈ 0.03), together with a full FLOPs accounting: 5–21% fewer FLOPs per training forward pass and 8–29% fewer FLOPs per rollout step at inference (Reviewers kKB6, 9eay)
  • A robustness argument for the PAdaNorm ablations: no tested configuration is a "bad" architectural choice, showing the method is robust to design choices (Reviewer kKB6, W3)

We hope that our responses and revisions resolve the concerns raised, and we would be happy to address any further questions.

Thank you for your time and effort.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.