- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an adaptive spatial latent representation with a reasonable architectural design. Its main idea is to separate latent feature storage from attention computation. The experiments show promising results on a few tasks. However, there is still not enough evidence that the method outperforms existing neural operators in accuracy or efficiency. Its generalization across different discretizations also remains unclear.
Critical strengths
The main strength of FST is its architectural design. It separates where latent features are stored from where global attention is performed. This gives the model more flexibility in how it represents and processes spatial information. Another interesting aspect is that the anchor positions can adapt to each input instead of staying fixed. I think combining these two ideas is a reasonable way to build a more flexible spatial representation. The overall design makes sense, even though its practical advantages still need stronger experimental evidence.
Critical weaknesses
My main concern is that the experiments do not provide enough evidence to justify the additional complexity of the proposed architecture. Most comparisons focus on prediction accuracy and parameter count. The paper reports some training time and GPU memory usage, but there is no systematic comparison of FLOPs, inference speed, or scalability. FST introduces several extra components, including anchor prediction, RBF interpolation, and recursive attention. All of these come with additional computational costs. Without a fair comparison under similar computational budgets, it is hard to tell whether these components are actually worth the added complexity. It would also be useful to see how the computational cost changes as the number of anchors, evaluation points, and input observations increases.
Another important issue is that the experiments do not clearly show why the proposed architecture is better than simpler spatial latent Transformers. The current ablations show some improvements, but they change multiple things at once. For example, replacing the function-space interaction with direct anchor self-attention also changes the number of tokens and the overall computation. This makes it difficult to identify where the improvements really come from. The benefits of adaptive anchors are also not consistent across different settings. More comparisons with recent spatial latent architectures would help. In particular, I would like to see whether adaptive anchors and function-space interactions still provide meaningful improvements when the models have comparable FLOPs, memory usage, and training budgets.
Questions for authors
-
Does the additional architectural complexity actually lead to better accuracy-efficiency trade-offs than recent neural operators, under a fair computational budget?
-
Does the model learn a physically meaningful spatial representation, or does it simply benefit from additional flexibility in latent parameterization?
3 comments
Thank you for the detailed response and the additional experiments. The new results address many of my concerns about computational efficiency and experimental fairness. They make the paper more convincing from an experimental perspective. However, my main concerns about architectural novelty and scientific understanding remain largely unchanged.
My biggest concern now comes from the new results on adaptive anchors. With 256 anchors, the fully adaptive design reduces the error by about 4.9% compared to the best uniform layout. With 1,024 anchors, the uniform layout performs about as well as the adaptive one. This suggests that the benefits of adaptive anchors are limited to certain settings. If adaptive anchors are not a major source of the performance improvement, it is unclear why they are presented as a central contribution and even highlighted in the title.
I also remain concerned about generalization across different discretizations. The authors suggest interpolating randomly sampled inputs back to the training points. This indicates that the trained model is still dependent on the input discretization used during training. Being able to accept arbitrary input coordinates does not necessarily mean that the model generalizes well across different discretizations.
Overall, I appreciate the effort the authors have made to improve the experimental evaluation. However, the new results do not substantially strengthen the architectural novelty or provide a deeper understanding of why the proposed design is better than existing alternatives. Comparisons with the most closely related spatial latent architectures are still limited, and the necessity of adaptive anchors remains unclear. For these reasons, I would keep my original rating.
Thank you for reading it closely.
Uniform placement can be seen as a special case of adaptive anchors. Adaptive anchors show a clearer improvement when anchors are scarce and length scales adapt as well (Section 4.2). In practice, we use adaptive anchors by default.
On input discretization, our earlier remark "For random subsets, users should first interpolate the inputs to the training points" was stated too broadly. Interpolation was only a suggestion when testing models that are not specifically trained on random subsets. Table 20's Direct column shows the performance on different discretizations at the inference stage (without retraining), while the Interpolated column shows the performance when interpolating back to the original input grid to keep the number of input points fixed. Direct FST stays robust across different discretizations without interpolation: its error is nearly unchanged from 1,424 down to 356 Burgers inputs (0.0461 to 0.0464, whereas Transolver rises from 0.0607 to 0.2677).
Interpolation only helps the test on Burgers when the input points are randomly chosen, because all models were only trained on evenly spaced 1D inputs. When testing with random subsets, UPT and Transolver show the same performance drop without interpolation, while FST actually keeps the lowest error. Below is the comparison with UPT and Transolver.
| Burgers input | Points | FST (Direct / Interpolation) | UPT (Direct / Interpolation) | Transolver (Direct / Interpolation) |
|---|---|---|---|---|
| Training inputs | 1,424 | 0.0461 ± 0.0062 / - | 0.0538 ± 0.0068 / - | 0.0607 ± 0.0005 / - |
| Every 2nd value | 712 | 0.0462 ± 0.0062 / 0.0460 ± 0.0062 | 0.0538 ± 0.0069 / 0.0538 ± 0.0068 | 0.1460 ± 0.0403 / 0.0607 ± 0.0006 |
| Every 4th value | 356 | 0.0464 ± 0.0065 / 0.0471 ± 0.0061 | 0.0545 ± 0.0069 / 0.0543 ± 0.0066 | 0.2677 ± 0.0453 / 0.0615 ± 0.0011 |
| Every 8th value | 178 | 0.0550 ± 0.0052 / 0.0573 ± 0.0046 | 0.0570 ± 0.0069 / 0.0561 ± 0.0064 | 0.4017 ± 0.0532 / 0.0668 ± 0.0018 |
| Random half | 712 | 0.0766 ± 0.0053 / 0.0467 ± 0.0059 | 0.0871 ± 0.0064 / 0.0538 ± 0.0069 | 0.1577 ± 0.0328 / 0.0610 ± 0.0006 |
| Random quarter | 356 | 0.1036 ± 0.0046 / 0.0502 ± 0.0061 | 0.1085 ± 0.0100 / 0.0547 ± 0.0073 | 0.2792 ± 0.0557 / 0.0626 ± 0.0016 |
Thank you again for the constructive discussion.
Thank you for the thorough review.
Summary
For outputs, a single trained FST keeps an error near 0.046 on every output grid from 32 × 32 to 201 × 1024, while Transolver's error rises from 0.061 to 0.136 on the coarsest grid (Table 3). For inputs, the newly added Table 20 tests trained FST on input subsets without retraining. With every fourth Burgers input value, FST's error is nearly unchanged. With a 64 × 64 Darcy input, FST's error increases by 5%. Randomly chosen subsets are harder, because FST saw only evenly spaced inputs during training. For random subsets, users should first interpolate the inputs to the training points.
Weaknesses
We agree. The revision adds a cost study in Appendix E. Table 27 reports FLOPs, inference time and memory for every Burgers and Darcy model. Figure 13 shows how FST's cost changes with the number of anchors, evaluation points, inputs and queries. Figure 14 compares model accuracy at equal training cost. Above a fixed base, FST's FLOPs grow linearly with inputs, queries and evaluation points. Predicting anchor coordinates accounts for only 1-3% of the FLOPs. Our answer to your first question below summarizes the findings.
We agree. Replacing the function-space update with self-attention over the anchors also changes the token count and removes the write-back. We therefore added a matched variant that changes only the field evaluation. Instead of evaluating the latent field at 512 points, the variant uses 512 learned tokens that read the anchors by cross-attention, like inducing points in Set Transformer. The rest of the update, token count and training protocol stay the same. The variant has 3.5% more parameters and about 3% fewer FLOPs. Evaluating the field lowers the error by 28% (0.0441 vs. 0.0614 over five seeds, Table 5).
We agree, and three seeds now show when adaptive anchors help. Adaptive anchors help with few anchors and when FST adapts length scales alongside coordinates. With 16 × 16 anchors, the fully adaptive design gives the lowest error (0.0486 vs. 0.0511 for the best uniform layout). With 32 × 32 anchors, a uniform layout is about as accurate as adaptive anchors. Section 4.2 now states these conditions instead of a general benefit.
We added the Universal Physics Transformer (UPT, NeurIPS 2024) as a baseline on Burgers and Darcy. We trained UPT under the protocol of the other baselines, with the same parameter budget and three seeds. FST has the lower mean error on both problems: 0.0461 vs. 0.0538 on Burgers and 0.0270 vs. 0.0334 on Darcy (Tables 3 and 7).
Questions
Compared with Transolver, yes. At equal training cost, FST has lower error than Transolver throughout training on Burgers, and after about the first two GPU-hours on Darcy. At inference FST runs more than ten times faster than Transolver on the full Burgers grid at batch size 1.
Compared with FNO, the answer depends on the setting. On Burgers, FST reads only 1,424 input values and trains on 2,500 sampled output points, so each update is cheap. FST reaches lower error than FNO for training budgets between roughly one and four GPU-hours and uses about a fifth of FNO's memory. With longer training, FNO becomes more accurate. On Darcy, every grid value is an input, so FST needs far more FLOPs and memory than FNO, though FST ends up more accurate (0.0270 vs. 0.0326 over three seeds). So FST gives the better trade-off for sparse inputs with predictions at selected points, and FNO gives the better trade-off on dense regular grids. At test time, localized queries can also decode from 16 nearby anchors on Burgers with little loss in accuracy (Section 4.4). The new limitations paragraph in Section 5 states this trade-off.
Input adaptation and extra flexibility both play a role. The learned layouts clearly depend on the input. On ImageNet the anchors gather on the object (Figure 12), and on Darcy the anchors shift from one input to the next (Figure 10). The extra flexibility also helps with few anchors, as in the 16 × 16 result above.