- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
StrataTok is a dynamic tokenizer for PDE forecasting from sparse and irregular observations. For each visible observation, it estimates local variation from its nearest spatiotemporal neighbors and uses this score to split the observation's mass between a global token bank and a variation-enriched token bank. Within each bank, tokens are formed by Transolver-style soft assignment and mass-normalized pooling. The authors evaluate it against five tokenizers under a shared forecasting backbone on nine simulated PDE settings and four RealPDEBench systems, report the lowest RMSE in all thirteen settings, and add robustness studies over observation density, sampling pattern, and token budget, plus a routing ablation.
Critical strengths
1. Meaningful and practically relevant problem setting. Most existing PDE tokenizers (Transolver, MeshTok, MATEY) assume dense or fully observed discretizations, while real measurements such as PIV or sensor networks are often sparse and irregular. Moving token formation into this regime is a worthwhile direction, and it is the main reason I see value in this line of work despite my concerns.
2. Simple, interpretable design with broad empirical coverage. The method is lightweight. It adds a parameter-free, neighbor-based variation score on top of standard soft assignment, which makes it easy to understand and to plug into existing architectures. The evaluation covers 1D and 2D, planar and spherical domains, and real experimental data from RealPDEBench, and it includes sweeps over observation density, sampling pattern, and token budget. This breadth makes the empirical claims at least testable, even though, as discussed below, I am not yet convinced by the baselines and protocol.
Critical weaknesses
1. Insufficiently strong and validated baselines. The "single learned bank" control (Transolver-style soft assignment) in Table 2 already outperforms all five Table 1 baselines on the three evaluated datasets. Relative to this stronger control, StrataTok reduces RMSE by only 7.7–10.3%, substantially narrowing the headline margins. Yet this control is absent from the main 13-task comparison. Furthermore, MeshTok and MATEY are evaluated through custom sparse adapters whose fidelity to the original methods is unclear, while relevant cross-attention tokenizers (e.g., Perceiver-style or AROMA-inspired encoders) are missing. These omissions make it difficult to establish the magnitude of the improvement over strong alternative tokenization mechanisms.
2. Limited external validation and statistical support. Although the datasets originate from established benchmarks, the paper uses custom splits, sparse-observation protocols, forecasting horizons, and metric aggregation, without reproducing a published evaluation protocol to anchor the results. The additional DINo/STRIDE comparisons also follow this custom setup. Established sparse forecasting evaluations, such as AROMA's Navier–Stokes and Shallow-Water experiments at 5%, 25%, and 100% observation density, provide a relevant reference. Moreover, Table 1 reports geometric means over only three runs without uncertainty estimates, while near-ties of 0.01% and 0.10% are counted as wins. Consequently, the claimed superiority across all thirteen settings is insufficiently supported statistically, and its comparability with published results remains unclear.
Questions for authors
-
Strength of the single-bank baseline. The single learned bank in Table 2 outperforms every Table 1 baseline on all three evaluated datasets. Why was this stronger control excluded from the main comparison? Can the authors provide its results across all 13 settings to establish the improvement attributable to variation-guided routing?
-
Fidelity and completeness of baseline implementations. How closely do the custom sparse MeshTok and MATEY adapters preserve the original methods' tokenization mechanisms? What evidence supports their fidelity? Additionally, are results available for sparse-input cross-attention tokenizers, such as Perceiver-style or AROMA-inspired encoders, under the same forecasting backbone?
-
Comparability with established evaluations. Given the custom splits, forecasting horizons, and metric aggregation, how can the reported results be compared with published sparse PDE forecasting benchmarks? Can the authors provide results under an established protocol, such as the Navier–Stokes and Shallow-Water sparse-observation evaluation used by DINo/CORAL/AROMA?
Thank you to the authors for conducting the additional experiments and providing detailed results. However, the evaluation still lacks a directly comparable assessment under an established forecasting protocol. Therefore, my main concern regarding the comparability and external validity of the results remains unresolved.