acceptodds
Under review as a conference paper at ICLR 2027

Does Transolver really need a Transformer?

Abstract

The widely used Transolver family of neural operators is based on physics-attention, which softly assigns the points of an unstructured mesh to a small number of slices, applies self-attention among the resulting tokens, and broadcasts the result back to the points. We provide a comprehensive empirical and theoretical analysis to elucidate the mechanisms which are responsible for model performance. To this end, we perform careful ablations on a challenging suite of nine 3D fluid dynamics benchmarks to find that replacing token attention with a constant linear map does not affect the accuracy. Thus, Transolver does not need a Transformer at all. However, removing the global mixing (slicing/deslicing) or doing it only once leads to performance collapse. We leverage the theory of averaging neural operators to explain and corroborate our findings by showing that just slicing/deslicing, in conjunction with pointwise MLPs, already suffices for universal approximation of continuous operators and attention is redundant in this context. Finally, we provide a novel FlashAttention-style efficient implementation of the key slicing/deslicing module of Transolver. This flashslice kernel streams slice and deslice over the points without materializing the heavy slice-weight tensor, while reproducing the best available implementation to floating point error. At the same time, it leads to very significant memory and compute savings, particularly at large slice counts.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, Transolver, physics-attention, operator learning, computational fluid dynamics, efficient GPU kernels, partial differential equations

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GFrvOpenReview ↗
Rating
4(+3)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper analyzes physics-attention, the core layer of Transolver.

On the theory side, the paper makes three claims:

A stack with a constant core is universal (Prop. 1), shown by reduction to the averaging neural operator (ANO) of Lanthaler et al. (2023). Pointwise stacks cannot approximate nonlocal operators (Cor. 1). An upper bound limits how many nonlocal functionals pass through one coupling (Prop. 2).

Finally, the paper presents FlashSlice, fused Triton kernels that never materialize the N×G slice-weight tensor.

Critical strengths

A controlled ablation at industrial scale. All variants share one training pipeline across nine large 3D aerodynamic and turbulence configurations, with up to 1.3×10⁸ points per sample. Each ablation is checked mechanically through checkpoint-key and bit-level freeze tests (App. C.2). The token-attention ablation stays at about 1×, while the coupling ablations reach up to 5.6× (MlpOnly) and 6.8× (SliceOnce). This gap is consistent in direction across all nine configurations and gives practitioners a usable signal about a widely deployed architecture.

A carefully validated kernel. FlashSlice is checked against an fp64 reference, is bit-deterministic, and is benchmarked against compiled and checkpointed baselines (Tables F.4, F.5). Its memory use is flat in G (Table F.3), which is a practical benefit for large-mesh training.

Critical weaknesses

(a) The empirical finding repeats Hu et al. (2026). The headline result is that token attention inside physics-attention can be removed without loss of accuracy. Hu et al. (2026, "Transolver is a Linear Transformer", AAAI) already reported this: they recast physics-attention as linear attention, removed the slice attention, and proposed LinearNO on that basis. The authors acknowledge this.

What this submission adds is mainly scale (nine 3D benchmarks) and a few more ablations of the coupling. Those additional results do not lead to a qualitatively new conclusion. "The attention is redundant; the global aggregation carries the accuracy" was already the message of prior work.

The one place where the paper takes a different position is its disagreement with Hu et al. on why the attention is redundant (projection tying). That argument rests on a single 2×2 grid on one benchmark (Fig. 2a), in which all four cells lie within about 1% of each other.

(b) The theory does not explain the experiments it is meant to explain.

Prop. 1 is a direct application of an existing theorem. It applies the ANO universality theorem of Lanthaler et al. (2023): setting all slice logits to zero makes every token the global mean, so the result follows. The construction does not use the slice structure, the learned partition of unity, or G at all. It would apply equally to any architecture that contains a global mean and pointwise MLPs.

Prop. 1 cannot separate the models being compared. As the authors themselves say (App. A.2, L756–764), the class with attention and the class without it are both universal. The result cannot distinguish BASELINE from NoTokenAttn, which is the comparison the paper is about.

Cor. 1 is close to trivial. It says that a composition of pointwise maps is pointwise.

Prop. 2 is loose by the authors' own account. It is a counting bound that the authors describe as "rather loose" (L328), and they state that it does not explain the size of the SliceOnce penalty (L821–829).

The links to the experiments are mostly hedged and qualitative. Examples include "suggestive of some of the other observations" (L318), "makes this observation unsurprising" (L302), and "the magnitude is not something we derive" (L829).

Questions for authors
  1. Will the FlashSlice code be made available to reviewers during the discussion period?

2.Apart from the slice/deslice operator of Transolver, what is new in the kernel design compared with FlashAttention and the tiled recomputation in Transolver-3?

8 comments

Thank you for a detailed and precise review. We agree with much of it.

(a) The empirical finding and Hu et al. (2026)

You are correct in saying that our disagreement with Hu et al. rested on a single grid. We extended the 2×2 factorial of attention × tying to a second benchmark (Table A of the general response). Every cell again lies within 0.7% of the Baseline, so the conclusion of the paper stands: tying is orthogonal to the redundancy of the attention.

The other differences from Hu et al. are listed in item 1 of the general response.

(b) The theory

We accept your four points: Proposition 1 applies the ANO theorem directly, it cannot separate Baseline from NoTokenAttn (as Appendix A.2 states) nor is it inteneded to, Corollary 1 is elementary, and Proposition 2 is a loose counting bound. Our original presentation could potentially be misread in how the theory explains the numerical evidence. In the revision we will:

  • state the theory's role as delimiting what expressivity can and cannot account for;
  • move the table of what the theory settles and what it leaves to experiment (Table A.1) into the main text;
  • drop "explain and corroborate" from the abstract.

Within that scope, the theory makes three predictions, which the new runs test:

  1. Corollary 1: no depth repairs MlpOnly. On SHIFT-SUV surface its gap is 1.58×, 1.61× and 1.56× at L = 4, 8 and 16.
  2. Proposition 2(ii): the nonlocal budget grows with the coupling rounds. The K-sweep is flat on SHIFT-SUV surface and monotone on DrivAerML surface (item 2 of the general response).
  3. Proposition 1: a constant core loses nothing in expressivity. A learned constant G×G core matches the attention on DrivAerML (0.984×).

Q1: Code availability

Yes. An anonymized repository is available at https://anonymous.4open.science/r/flashslice-review-557F/. It contains the FlashSlice kernels, the layer with every ablation flag, the tests and the benchmark scripts. We will release the code publicly on acceptance.

Q2: What is new in the kernel?

Against FlashAttention:

  • Slice is the transpose of attention's reduction: it accumulates per slot over points while the softmax runs over slots, so each point's normalizer must be known before its contribution is added.
  • A tied coupling pays for the membership once: the deslice hands its softmax statistics to the slice.
  • The learned temperature's gradient is the most rounding-sensitive quantity, so the precision of each dot is chosen per mode (Appendix F).
  • No atomics are used, so runs are bitwise deterministic.

Against Transolver-3:

their tiling is a schedule in PyTorch that trades time for memory. FlashSlice is a fused kernel that lowers both. The two do not conflict: the kernels can run inside each tile. One layer, forward and backward, at 4.2M points:

implementationtimepeak memory
eager525 ms44.1 GB
FlashSlice174 ms36.1 GB
Transolver-3 tiling (100k-point tiles)698 ms24.1 GB
FlashSlice inside Transolver-3's tiling284 ms24.1 GB

Inside the tiling, FlashSlice keeps the tiling's memory at 2.5× its speed, and matches the eager layer to 6×10⁻⁷ (output) and 4×10⁻⁶ (gradients), relative. We will add this comparison to Section 5.

Thank you for the response.

  • Novelty. The main finding, that token attention can be removed without loss, was already reported by Hu et al. (2026). The new tying experiment addresses only a secondary point.
  • Theory. The authors accept all four points, and the new runs do not change this. Cor. 1 holds by construction. Prop. 2 is an upper bound, consistent with both a flat and a monotone K-sweep. Prop. 1 does not predict trained accuracy.
  • Kernel. The comparison with Transolver-3 is new to the rebuttal and covers a single layer at one size. Inside the tiling, FlashSlice gives a 2.5× speedup but does not lower the memory floor, so it does not extend the trainable mesh size. This is solid engineering, but not a new idea.

The revision would substantially change the paper's claims, so I maintain my rating.

AuthorsOpenReview ↗

Response to Reviewer GFrv's follow-up (1/2): Novelty and Theory

We thank the reviewer for their prompt reply and respect the opinion of the reviewer. However, we wish to rebut the points that you have raised below:

  1. Novelty: We had already acknowledged in the submitted version of our paper that the Hu et. Al. had pointed out that attention is not needed in Transolver and we have repeated this point multiple times in our rebuttal. However, their demonstration was on very small-scale 2-D experiments and lacked a coherent explanation of why this is the case. Our paper has been written to precisely address these limitations. We consider large-scale 3D industrial benchmarks and find that token-attention is not necessary. More crucially, we have proposed a coherent explanation of why token attention can be removed without any loss of accuracy. We have used both empirical as well as theoretical means to do so and these provide a much deeper explanation to the reader as to what is essential in the Transolver model and what is not. Interpreting the reviewer’s conclusion as “once a phenomenon is observed, there is no further need to investigate and explain it” runs counter to the standard practice in science where the purpose of many further studies is to precisely find such explanations of new observations. We would appreciate if the reviewer points out genuine issues with our explanation but dismissing them as secondary does not constitute a point that we can constructively engage with.
  2. Theory: We have repeatedly stated that Proposition 1 describes the limits of what is expressible by a certain neural operator model. Using it, clearly shows that token attention is not necessary for expressing relevant classes of operators and adding or removing token attention should not change what operators a model can approximate and what it cannot. We believe that this already provides an explanation of why token attention is redundant. Asking for precise bounds on trained models in this context is unrealistic as no such bounds exist in the literature, to the best of our knowledge. We would be happy to add references to such precise estimates if the reviewer can kindly provide them. Moreover, we would like to reemphasize that, although it is simple in its formalism and calculations, our theory explains several aspects of the observed empirical results, providing the reader with a coherent explanation resting on empiricism, heuristics and theory. We believe that such an approach is fruitful in this domain, rather than looking for precise estimates which are unrealistic to obtain, given current tools.
Reviewer_GS5pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies which architectural components are responsible for the effectiveness of Transolver in neural operator learning. Through systematic ablation studies on nine 3D fluid dynamics benchmarks, the authors find that token self-attention can be replaced by simple linear transformations with little impact on prediction accuracy, whereas removing slice/deslice-based global interactions or restricting them to a single stage leads to substantial performance degradation. The authors further interpret the slice/deslice mechanism as a low-rank nonlocal operator and use existing universal approximation theory to explain why token attention is not essential for expressivity. Finally, they introduce FlashSlice, a fused GPU implementation that avoids materializing slice weights, significantly reducing memory consumption and improving computational efficiency, particularly for large slice counts.

Critical strengths
  1. The paper conducts extensive ablation studies across nine 3D fluid dynamics benchmarks, with controlled comparisons of token attention, slice/deslice coupling, and other architectural choices. The consistent results provide strong empirical evidence that global coupling, rather than token self-attention, is the primary contributor to Transolver's performance. This is a valuable finding for understanding and simplifying a widely adopted neural operator architecture.
  2. The proposed FlashSlice implementation addresses a major computational bottleneck by eliminating the need to materialize large slice-weight tensors. The authors demonstrate meaningful improvements in training speed and memory efficiency, supported by numerical accuracy checks and evaluations at different problem scales. This provides practical value beyond the architectural analysis, particularly for large-scale scientific simulations.
Critical weaknesses
  1. The observation that token self-attention is unnecessary for Transolver has already been reported by Hu et al. (2026, Transolver Is a Linear Transformer), which also proposes an attention-free alternative. Although this paper provides broader empirical validation and additional theoretical analysis, the latter largely builds on existing universal approximation results. Consequently, the novelty of the paper's central scientific contribution is somewhat limited, weakening its case for acceptance at ICLR.
  2. The paper concludes that repeating slice/deslice operations at every layer is essential, primarily based on the poor performance of SliceOnce. However, this variant simultaneously removes repeated global coupling and changes the persistent pointwise processing and decoding structure, as acknowledged in Appendix A.4. Therefore, the observed degradation cannot be attributed specifically to the absence of repeated coupling. This weakens one of the paper's main architectural conclusions. A controlled ablation preserving the full pointwise stream while varying only the frequency of global coupling would be necessary.
Questions for authors
  1. Hu et al. (2026) have already demonstrated that token self-attention can be removed from Transolver without substantially degrading accuracy, while Yang et al. (2026) provide a related low-rank operator interpretation. Could the authors clarify what fundamentally new understanding this paper establishes beyond these studies? In particular, does the theoretical analysis provide new insights or predictions beyond applying existing universal approximation results to Transolver?
  2. The SliceOnce ablation simultaneously removes repeated slice/deslice operations and changes the persistent pointwise processing and decoding structure. How can the authors distinguish the effects of these changes? Could they provide a controlled comparison that preserves the original pointwise stream and varies only the number or frequency of slice/deslice operations? This is important for establishing whether repeated global coupling is truly necessary.
5 comments

Thank you for the constructive review. Both of your questions are answered in the general response; we add only what is specific to your review.

Q2 / W2: Does repeated coupling matter, with the point stream held fixed?

The reviewer is correct in claiming that SliceOnce removes the repeated coupling and the persistent pointwise processing together. It also reads out affinely, with no pointwise decoder, and slices with a single head.

We ran the controlled ablation you proposed (item 2 of the general response), and it changes one of our conclusions:

  • Most of SliceOnce's deficit was due to the missing pointwise decoder.
  • Whether repeated coupling matters depends on the nonlocality of the task.

We will replace the claim "slicing/deslicing has to be repeated every layer" with this finding, and state SliceOnce's differences from the Baseline explicitly.

Q1 / W1: What is new beyond Hu et al. (2026) and Yang et al. (2026)?

  • Hu et al. (2026): please see item 1 of the general response.
  • Yang et al. (2026, LRSA) frame global mixing as a compress–process–reconstruct template and build a new architecture whose process step is standard attention. They do not ablate Transolver's components. Our results show that for Transolver, the process step does not need to be attention, or content-dependent at all.

Thanks to the authors for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns, particularly for the new controlled ablations that preserve the pointwise stream while varying the number of coupling rounds. These results effectively address my second concern, which also provide a more nuanced understanding of the role of repeated global coupling. Besides, I appreciate the authors' willingness to revise the corresponding claims.

Regarding novelty, the response helps clarify the additional contributions beyond Hu et al. (2026), particularly the broader empirical analysis and the practical value of FlashSlice. However, I still have some reservations about the novelty of the central scientific findings, as the redundancy of token self-attention and the importance of slice/deslice operations have already been identified in prior work. The theoretical analysis, while useful, also largely builds on existing results.

Overall, the rebuttal has improved my assessment of the paper. Nevertheless, given the remaining concerns about novelty, I would line to maintain my current rating. Indeed, my assessment is close to the acceptance threshold, and I will not strongly oppose acceptance.

AuthorsOpenReview ↗

Reply to the reviewer.

We start by thanking the reviewer for their reply and for their positive assessment of our paper. We would like to point out that we have now uploaded a revised version that implements the changes that we had outlined during the rebuttal. We hope that the revised version addresses the remaining concerns of the reviewer.

With respect to novelty, we had already acknowledged in the submitted version of our paper that the Hu et. Al. had pointed out that attention is not needed in Transolver and we have repeated this point multiple times in our rebuttal. However, their demonstration was on very small-scale 2-D experiments and lacked a coherent explanation of why this is the case. Our paper has been written to precisely address these limitations. We consider large-scale 3D industrial benchmarks and find that token-attention is not necessary. More crucially, we have proposed a coherent explanation of why token attention can be removed without any loss of accuracy. We have used both empirical as well as theoretical means to do so and these provide a much deeper explanation to the reader as to what is essential in the Transolver model and what is not. Thus, we believe that the current paper serves an essential purpose: to explain what the key ingredients in a very popular AI model in this domain are and what is not essential. We sincerely hope that this will spur further research on better Transolver variants.

Given the above points, we kindly request the reviewer to upgrade their positive assessment of our paper by raising the score to a positive form of acceptance.

Reviewer_aMhGOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Through extensive ablations on nine 3D fluid dynamics benchmarks, the authors show that Transolver's token self-attention can be replaced with a constant linear mapping without sacrificing accuracy. Using Averaging Neural Operators (ANOs), they theoretically establish the importance of repeated slicing/deslicing and pointwise MLPs for universal approximation. To improve efficiency, they propose FlashSlice, an IO-aware Triton kernel that accelerates training by up to 6.5× and reduces memory usage by up to 96% while preserving numerical precision.

Critical strengths

S1:The paper conducts extensive ablation studies on various Transolver variants across nine benchmarks, empirically demonstrating the necessity of each individual design component in Transolver.

Critical weaknesses

W1:The paper only demonstrates the necessity of individual architectural components through experiments, but does not leverage these findings to establish general design principles or propose a concrete design methodology. W2: Proposition 1 only proves that attention is redundant in terms of representational capacity, but does not explain why removing attention leads to performance gains. Theoretically, redundancy does not necessarily imply a decline in performance, as the existing performance could be maintained even if the redundant component were removed. W3: The observations that removing attention from Transolver can improve performance and that Slice/DeSlice operations are the primary source of performance gains are not novel, as they have already been discussed in LinearNO[1]. The main contribution of this paper lies in providing a theoretical explanation for these observations. However, the paper claims to offer a theoretical justification for removing attention, while the theory presented in W2 does not appear to adequately support this claim. This further weakens the paper's overall contribution.

Questions for authors

Q1: Proposition 1 claims that attention is redundant in terms of representational capacity. Meanwhile, Tiwari et al.[2] replaces the attention mechanism in Transolver with Mamba and also reports performance improvements. From a theoretical perspective, both attention and Mamba can enhance a model's nonlinear approximation capabilities and potentially increase its performance, rather than necessarily lowering its performance. Q2: Proposition 2 in the paper suggests that SliceOnce leads to a performance degradation. However, Wang et al.[3] introduces physical cross-attention, which can theoretically be viewed as analogous to the Slice/DeSlice mechanism. Its architecture similarly employs a single physical cross-attention operation followed by stacked attention layers, making it appear structurally similar to the SliceOnce implementation proposed in this paper. Nevertheless, it achieves substantially better performance than Transolver. Based on the concerns raised in Q1 and Q2, I have reservations about the validity of the experimental results and the soundness of the theoretical analysis. I would appreciate further clarification from the authors on these issues.

Ref: [1] Hu, Wenjie, et al. "Transolver is a linear transformer: Revisiting physics-attention through the lens of linear attention." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 1. 2026. [2] Tiwari, Karn, Niladri, Dutta, N M Anoop, Krishnan, Prathosh, Ap. "Latent Mamba Operator for Partial Differential Equations." Proceedings of the 42nd International Conference on Machine Learning. PMLR, 2025. [3] Wang, Tian, and Chuang Wang. "Latent neural operator for solving forward and inverse pde problems." Advances in Neural Information Processing Systems 37 (2024): 33085-33107.

4 comments

Thank you for the review. We answer each point below.

W2: "Why does removing attention lead to performance gains?"

We have not claimed that removing the attention improves accuracy. Rather, the claim is that removing attention does not lead to any significant loss of accuracy: NoTokenAttn lies within −2.6% to +7.9% of the Baseline on the nine benchmarks. Proposition 1 is used for one thing only: to show that removing the attention costs no expressivity. We will reword the sentences that could potentially be read as claiming any gains over attention.

W3: Novelty relative to LinearNO

Please see item 1 of the general response.

W1: Design principles

The practical recommendations will move into the main text:

  1. Drop the token attention. Content-dependent mixing gave nothing on any benchmark (Q1 below).
  2. Keep slice/deslice and the full-resolution point stream, including a pointwise decoder.
  3. Set the number of coupling rounds by the task's nonlocality (item 2 of the general response). The MlpOnly penalty is a cheap pre-test.
  4. Run the coupling with FlashSlice, inside Transolver-3's tiling when memory binds.

Q1: Mamba (Tiwari et al., 2025)

LaMO changes the encoder, the decoder and the token mixer together, so its gains cannot be attributed to the state-space model. To isolate the mixer, we replaced the token attention in our scaffold and kept everything else. DrivAerML surface, where removing the attention leads to the largest errors; the last three rows are means over two seeds:

token mixerratio to Baseline
softmax attention (Baseline)1
identity (NoTokenAttn, A = I)1.047×
bidirectional selective SSM (Mamba-style)1.017×
token-mixing MLP0.987×
learned constant G×G core0.984×

The SSM does not improve on attention, and a constant matrix does as well as either. Any operation on the G tokens only re-parameterizes the core A in Eq. (1).

Q2: LNO (Wang & Wang, 2024)

LNO decodes through a pointwise MLP, which our SliceOnce lacked. With a 4-block pointwise decoder, which is LNO's structure, SliceOnce's penalty falls from 2.87× to 1.28× on SHIFT-SUV and from 6.31× to 3.02× on DrivAerML. Coupling more often removes the rest (item 2 of the general response). LNO's design is therefore consistent with our analysis.

LNO's own comparison with Transolver is also mixed: better on four of its six 2D benchmarks, worse on Airfoil (8.5%) and Plasticity (2.4×).

AuthorsOpenReview ↗

Request to the Reviewer.

We would like to point out that we have now uploaded a revised version of paper, with changes highlighted in blue, that implements all the changes that we have outlined during the rebuttal. Given that the rebuttal and discussion period closes soon, we take the opportunity to request the reviewer to provide some feedback on whether our rebuttal and revised version have addressed your points of criticism. If so and in view of the positive assessments of our paper by other reviewers, we kindly request the reviewer to upgrade their assessment accordingly.

Reviewer_aMhGOpenReview ↗

Thanks for the detailed response.

Thanks for the detailed response and additional experiments. My concern about novelty is not fully addressed. I would keep my initial rating.

We request the reviewer to be more specific about what your particular concerns on novelty. It is impossible for us to address them unless you actually specify them. The fact that your review arrives a few hours before the end of the rebuttal period makes it nearly impossible for us to address them.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Answer Part

We thank the reviewers for their careful reviews. Two points recur across the reviews:

  1. the novelty of our findings relative to LinearNO (Hu et al., 2026);
  2. whether SliceOnce isolates the effect of repeated coupling.

We answer both here, with new experiments. All other points are answered in the individual response to each reviewer. Every ratio below is to a Baseline trained under the same conditions as the new runs.

1. Novelty relative to LinearNO (Hu et al., 2026)

Hu et al. remove the token attention and untie the projections, on small-scale 2D benchmarks with at most 32k points. Our paper ranges from large-scale industrial experiments, to a theory that aims to explain the experimental observations, to a highly-efficient kernel for the module that really matters for Transolver's performance. The differences at each step, with the results of this rebuttal marked New:

Large-scale industrial experiments (nine 3D benchmarks, up to 1.3×10⁸ mesh points per sample):

  • We test what Hu et al. only state. They never remove the slice/deslice step. We do: it costs 1.04× to 5.6×, more for more nonlocal operators, and it is not merely a model capacity effect. New: depth does not close the gap (1.58× / 1.61× / 1.56× at L = 4 / 8 / 16), and the number of coupling rounds a task needs follows the same ordering (item 2).
  • Tying is not what makes the attention removable, contrary to their explanation. New: on a second benchmark, every cell of attention × tying again lies within 0.7% of the Baseline (Table A).
  • No content-dependent mixing is needed. New: a learned constant G×G core, a token MLP and an SSM all match the attention within 2% (response to Reviewer aMhG, Q1).
  • LinearNO's layer is costly and no more accurate. New: it trains 2.4× longer than Transolver at matched latent width, and about 16× longer at its published width (Table B).

Theory. Hu et al. give a Monte Carlo convergence argument. Our Section 4 shows that expressivity needs the coupling and not the attention. New: the three predictions of our theory are now tested (response to Reviewer GFrv, part (b)).

Kernel. Hu et al. have no counterpart. FlashSlice makes slice/deslice, the module that underpins the model accuracy, up to 6.5× faster in up to 96% less memory. New: it composes with Transolver-3's tiling (response to Reviewer GFrv, Q2).

Table A. Token attention × projection tying. Change against the tied Baseline of each row; the LinearNO column is the authors' released layer in our scaffold.

NoTokenAttn (tied)Untied (with attention)LinearNO (untied, no attention)
SHIFT-SUV surface+0.5%−0.5%−0.4%
SHIFT-Wing surface−0.2%−0.7%−0.7%

Table B. Training wall clock on one four-GPU GH200 node, same pipeline within each benchmark.

benchmark (steps)TransolverLinearNO, latent width = GLinearNO, published latent width
SHIFT-SUV surface (50k)3.1 h7.4 h (2.37×)≈ 50 h (≈ 16×, projected), same accuracy
SHIFT-Wing surface (250k)11.7 h27.6 h (2.36×)—

2. Coupling frequency, with the point stream kept

GS5p is right that SliceOnce changes two things at once: it couples only once, and it drops the persistent point stream and with it the pointwise decoder. In the new runs, all 8 layers keep their pointwise MLP and only K of them keep the physics-attention sublayer, evenly spaced. We also add a 4-block pointwise decoder after SliceOnce's single deslice. Error is shown as a ratio to the Baseline:

variantcoupling roundsSHIFT-SUV surfaceDrivAerML surface
MlpOnly (K = 0)01.61×5.46×
K = 111.023×1.377×
K = 221.009×1.124×
K = 441.012×1.018×
Baseline (K = 8)811
SliceOnce (paper)1, no point stream2.87×6.31×
SliceOnce + pointwise decoder11.28×3.02×

Two conclusions follow:

  • Most of SliceOnce's penalty came from the missing pointwise decoder. Adding it cuts the penalty from 2.87× to 1.28× and from 6.31× to 3.02×, as Appendix A.4 anticipated.
  • The number of coupling rounds needed depends on the task, and tracks its locality gap. On SHIFT-SUV surface (MlpOnly 1.61×) a single coupling recovers almost everything. On DrivAerML surface (MlpOnly 5.5×) the error falls with every added round (1.38× → 1.12× → 1.02×), as Proposition 2(ii) predicts for operators with a large nonlocal part.

Revisions on these two points

  • We will replace the claim "slicing/deslicing has to be repeated every layer" with the task-dependent finding of item 2, and state SliceOnce's two differences from the Baseline.
  • We will add the second benchmark to the attention × tying grid, add the cost of LinearNO, and expand the comparison with Hu et al.

The other revisions are listed in the responses to the individual reviewers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.