- Rating
- 4(+3)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes DRAFT-Solver, a neural surrogate for steady-state PDEs on unstructured meshes. Each DRAFT block combines two attention pathways:
Coarse pathway. Nodes are softly assigned to K latent tokens through an assignment matrix computed once from the initial embeddings and shared across all blocks. Self-attention is applied among the tokens, and the result is routed back to every node. Fine pathway. A scoring network selects a fixed-budget subset of M = ⌊bN⌋ nodes, using top-M hard routing with a straight-through estimator. Full self-attention is applied among the selected nodes.
Global PDE parameters are injected through FiLM. An optional pairwise ranking loss encourages the scorer to give higher scores to nodes with larger prediction errors.
The method is evaluated on AirFRANS, ShapeNet Car, CMOS semiconductor simulations and SPACIER-M. Each benchmark reports a variant with ranking supervision and one without. The paper also analyses where the fine budget is allocated (Figs. 4–5, App. B–C).
Critical strengths
Broad evaluation with a single architecture. The same design is applied to 2D and 3D aerodynamics, semiconductor-device simulation and biomolecular electrostatics. The reported numbers on AirFRANS and CMOS are strong compared with the baselines listed.
An informative and candid analysis of the learned allocation. The visualisations of the selected nodes (Fig. 4; Figs. 6–9 for twelve samples per dataset) and the cumulative-capture curves (Fig. 5) show consistent, interpretable allocation patterns, such as surface-focused selection on ShapeNet Car. The discussion of when ranking supervision helps and when it hurts (Sec. 5, App. B.3–B.4) is open about the method's limitations.
Critical weaknesses
- Limited novelty
The coarse pathway is essentially Transolver's physics-attention (soft slicing into tokens, token attention, routing back with the same weights; Eqs. 3–4), and the fine pathway is standard top-k selection with a straight-through estimator. I do not see a distinctive new idea beyond combining these known components.
- Efficiency is never measured
Although the paper is motivated by the cost of attention on large meshes (L37–38), it reports no runtime, memory or parameter count for DRAFT or any baseline, so the central efficiency claim is unsupported.
- Missing ablations
There is no coarse-only model and no random-selection fine pathway, so there is no evidence that the learned node selection, rather than the extra attention layer, drives the gains.
- Headline results are selected per dataset
The contributions (L69–72) report, for each dataset, whichever variant (with or without ranking) scores better on the test metrics, without a validation-based selection rule. Neither variant is consistently better than the baselines.
Add. In my view, the text in Figure 1 of this paper is far too small, while the font size in Figure 4 is too large. I would like to see the variation in font sizes across the figures kept within a reasonable range; this is an aspect of the paper's overall quality.
Questions for authors
-
What are the runtime, peak memory and parameter count of DRAFT compared with the baselines?
-
How does DRAFT compare with a coarse-only model and with random node selection in the fine pathway?
5 comments
(Continuation of our previous comment.)
Novelty.
We agree that the coarse pathway builds on soft token aggregation in the spirit of Transolver, and that top-k selection with a straight-through estimator is a known tool. Our contribution is not a new attention primitive. It is a new answer to a central question for mesh-based surrogates: where to spend node-level computation.
- Existing solvers do not learn where to refine. Transolver treats all nodes uniformly through latent tokens. SpiderSolver prescribes the refinement regions from the geometry, through boundary-informed partitions.
- DRAFT learns it. To our knowledge, DRAFT is the first PDE surrogate that learns, end-to-end and per sample, a fixed-budget set of nodes for additional node-level attention, on top of a global token pathway that updates every node. No surfaces, partitions or masks are specified.
The new experiments show that this design choice, and not the individual components, is what matters:
- Learned allocation matters. At the same budget, random refinement is worse than learned refinement. Geometry-prescribed refinement is worse still (2× the CL error), even though it is the SpiderSolver-style prior and the natural choice for aerodynamics.
- One mechanism across domains. Without any dataset-specific masks, the same mechanism learns different, physically meaningful allocations across four domains (Fig. 4): the airfoil surface, especially the leading edge, in aerodynamics, the device interfaces in CMOS, and most atoms in SPACIER-M, where the target is a global sum.
- Lower cost. DRAFT achieves this while being faster than both Transolver and SpiderSolver, up to the full mesh.
Soft token aggregation and top-k selection are building blocks. Their combination into a learned, budgeted refinement for PDE surrogates is new, and we will make this distinction explicit in the revised paper.
Selection of the headline results.
Reporting both variants on every benchmark is intentional and not a selection procedure. Whether error-ranking supervision helps is one of the questions the paper studies. Its answer depends on the structure of the target, and it can only be observed by reporting both variants side by side.
- No hidden selection. Every table reports both variants, so the reader always sees the result of each one, including where ranking hurts.
- The outcome follows a consistent pattern (Section 5). Ranking helps when the metric is local and region-specific: on ShapeNet Car the surface drag depends on the surface region that ranking concentrates on. It does not help when the target aggregates outputs of different physical relevance (AirFRANS, CMOS) or is a signed global sum with cancelling contributions (the SPACIER-M solvation energy). This gives a criterion that can be applied a priori: use ranking when the downstream quantity is local, and end-to-end allocation otherwise.
We agree that the contribution statement (L69–72) mixes the two variants. In the revision, we will rewrite it to state the results of the default model explicitly and to present ranking supervision as an analysed extension, together with the criterion above.
Figure font sizes. Thank you, we will harmonize the font sizes, enlarging the text in Figure 1 and reducing it in Figure 4.
We hope that the new ablations and efficiency measurements address the reviewer's main concerns, and we would be grateful if the reviewer would consider them in the final assessment.
Follow-up
The current PDF does not yet include the changes promised in the rebuttal. As a reminder, ICLR allows authors to upload a revised PDF during the discussion period.
We thank the reviewer for the review. We will first answer the questions with all the ablations and efficiency measurements requested, and then go through the raised weaknesses. Unless stated otherwise, the experiments use AirFRANS with the configuration of the paper (no ranking loss, b = 0.09, K = 32) and change only the component under study.
Coarse-only model and random node selection.
The reviewer asked whether the gains come from the learned selection or simply from the extra attention layer. The ablation separates the two:
Efficiency: runtime, peak memory and parameter count.
Setup: single NVIDIA H200, AirFRANS sample subsampled to 32,000 nodes, batch size 1, 20 warm-up and 100 timed iterations. Training time is one forward + backward + optimizer step.
We also report the effect of the fine budget b on accuracy and cost:
All budgets outperform both baselines on all metrics. b = 0.09 gives the best accuracy–cost trade-off, and up to b = 0.20 DRAFT remains faster than Transolver.
(Continued in the next comment.)