- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses a meaningful problem and achieves promising results. However, its methodological novelty is incremental, and the experiments do not sufficiently validate the central claim that history-guided routing mitigates forecast smoothing. More targeted ablations and experimental analyses are needed.
Critical strengths
The proposed history-guided slice assignment provides a reasonable approach to reducing noise-driven feature mixing during diffusion denoising. The method achieves competitive forecasting results across five benchmark datasets. The additional spectral analyses provide useful evidence regarding temporal variation preservation.
Critical weaknesses
- Limited methodological novelty. HiSSA largely follows existing slice-based attention mechanisms. Its main contribution is history-guided slice assignment. The authors should clarify its differences and advantages over Transolver and Transolver++.
- Insufficient validation of the core motivation. The paper claims that noisy-state-based grouping causes dynamics mixing and forecast smoothing, but lacks direct evidence. A controlled comparison between history-guided and noisy-state-guided slice assignments is necessary.
Questions for authors
- Insufficient ablation and sensitivity analyses. The current ablations do not directly demonstrate HiSSA's contribution to frequency preservation. Additional frequency-domain comparisons, routing strategy ablations, and sensitivity analyses of the number of slices are recommended.
- Limited representativeness of visualization results. Figure 1 shows that DiCTS captures a sharp change at the final prediction timestamp, which appears potentially anomalous, while other methods fail to capture it. Although the appendix provides additional visualizations, more representative examples and clarification of the selection criteria would strengthen the conclusions.
- Missing computational efficiency analysis. The paper lacks comparisons of training time, inference time, and computational overhead, making it difficult to assess the practical efficiency of DiCTS.
- Inconsistencies in experimental results. The VS values for USHCN and MIMIC-III differ between Tables 1 and 3.
- Presentation issues. The abstract should more clearly emphasize forecast smoothing as the primary motivation. The Introduction should better explain Figure 1. In Figure 2, the output following the Head module is unclear, and the distinction between DiCT and DiCTS should be clarified. The conclusion would also benefit from a brief discussion of limitations and future work.
7 comments
Author Response 2: Detailed Response
W1. Limited Methodological Novelty
We thank the reviewer for the opportunity to clarify our contribution. Our contribution is noise-decoupled routing: identifying that the routing source of slice attention becomes a failure point inside a diffusion denoiser, and separating the organization of query interactions from the stochastic states being denoised.
Why Transolver-style routing is problematic in a denoiser. Transolver [1] and Transolver++ [2] compute slice weights from the same point features that are aggregated. In PDE solving these features are clean, and feature similarity reflects similar physical states. In a denoiser they are noisy query states: similarity is partly determined by the injected noise, so queries with distinct temporal dynamics can be grouped together, and because the same assignments govern aggregation and redistribution, mismatched groupings propagate in both directions.
Differences. HiSSA derives assignments only from query coordinate embeddings and the history context (Eqs. 4–5), obtained by a dedicated routing cross-attention to the history memory. History thus determines which queries exchange information, and the denoising states determine what is exchanged. Transolver++ adds a per-point adaptive temperature and a Gumbel-softmax reparameterization, which sharpen the assignments but do not change their source, so the noise dependence remains.
| Transolver [1] / Transolver++ [2] in a denoiser | HiSSA (Ours) | |
|---|---|---|
| Routing input | Features being aggregated (noisy query states) | Query coordinates and observed history |
| Access to history when grouping | Only indirectly, through the noisy states | Direct, via routing cross-attention (Eq. 4) |
| Dependence on injected noise and step | Yes | None (by construction) |
| Assignments during sampling | Recomputed for every step and sample | Computed once, reused across steps and samples |
Advantages (same backbone and content path; only the routing differs).
- Forecasting accuracy. History-guided routing reduces mean CRPS on all five datasets (by 5.9–26.5%) and mean VS on four of five (W2 & Q1 (1)).
- Slice attention alone is not sufficient. Noise-dependent routing has worse mean CRPS than standard self-attention on all five datasets, so directly transplanting Transolver-style slicing into the denoiser degrades performance; the gain of HiSSA comes from the routing source rather than from slice-based aggregation itself.
- Stable grouping under noise. Resampling the noise alone changes the grouping of noise-dependent routing by 10.4–67.9% of a random-permutation reference, whereas history-guided assignments are unchanged (W2 & Q1 (2)).
- Frequency preservation. Relative to noise-dependent routing, history-guided routing lowers the band-wise energy score in 12 of 15 dataset–band cells (by 0.7–5.4%), including the high band on four of five datasets, and the absolute log-amplitude error in 12 of 15. Noise-dependent routing raises band amplitudes relative to standard self-attention (14 of 15 cells) without a consistent gain in agreement with the observed amplitudes (7 of 15), whereas history-guided routing improves agreement in all 15 (W2 & Q1 (3)).
- Lower sampling cost than noise-dependent routing. Assignments are computed once per block and reused across denoising steps and forecast samples, which reduces counted sampling FLOPs by 7.00–10.75%, with 0.25–0.33M additional router parameters. The cost relative to standard self-attention is reported in Q3.
We have revised the manuscript to state these differences explicitly.
[1] Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for PDEs on general geometries. In ICML, volume 235, pp. 53681–53705, 2024. [2] Huakun Luo, Haixu Wu, Hang Zhou, Lanxiang Xing, Yichen Di, Jianmin Wang, and Mingsheng Long. Transolver++: An accurate neural solver for PDEs on million-scale geometries. In ICML, volume 267, pp. 41432–41449, 2025.
Author Response 3: Detailed Response
W2 & Q1. Insufficient Validation of the Core Hypothesis and Ablation/Sensitivity Analyses
We thank the reviewer for pointing out that our ablation lacked the noise-dependent routing control needed to test the core hypothesis directly, since we compared HiSSA only with standard self-attention. We have added (1) a controlled comparison of routing strategies, (2) a counterfactual noise intervention, (3) a frequency-domain comparison of the ablation variants, and (4) a sensitivity analysis of the number of slices.
1. Controlled comparison of routing strategies. We add a noise-dependent routing variant, which keeps the slice–attend–deslice block (Eqs. 6–8) but derives assignments from the noisy denoising states rather than from history and coordinates. Aggregation, slice attention, deslicing, and all training settings are identical; history-guided routing additionally contains the routing cross-attention of Eq. 4 (0.25–0.33M parameters), which acts only through the assignments. The variants are named as in Table 3 of the revised manuscript:
- DiCTS (Standard Self-Attn): (formerly "Ours w/o HiSSA"): standard query self-attention, without slice-based routing.
- DiCTS (Noise-dependent routing): slice-based attention with assignments derived from the noisy denoising states.
- DiCTS (Ours): slice-based attention with assignments derived exclusively from observed history and target coordinates.
Table. Ablation on five IMTS benchmarks (mean ± sample standard deviation over five seeds; lower is better).
| Model | Metric | HumanActivity | PhysioNet'12 | USHCN | MIMIC-III | MIMIC-IV |
|---|---|---|---|---|---|---|
| DiT | CRPS | 0.1133 ± 0.0015 | 0.2778 ± 0.0014 | 0.1657 ± 0.0054 | 0.2972 ± 0.0022 | 0.2399 ± 0.0013 |
| VS | 0.0358 ± 0.0005 | 0.0910 ± 0.0003 | 0.0999 ± 0.0098 | 0.1232 ± 0.0008 | 0.1193 ± 0.0015 | |
| DiCTS (Standard Self-Attn) | CRPS | 0.0981 ± 0.0025 | 0.2632 ± 0.0025 | 0.1617 ± 0.0009 | 0.2767 ± 0.0034 | 0.1939 ± 0.0010 |
| VS | 0.0325 ± 0.0006 | 0.0867 ± 0.0002 | 0.1031 ± 0.0095 | 0.1153 ± 0.0020 | 0.0972 ± 0.0030 | |
| DiCTS (Noise-dependent routing) | CRPS | 0.1128 ± 0.0023 | 0.2784 ± 0.0014 | 0.1678 ± 0.0048 | 0.2993 ± 0.0015 | 0.2444 ± 0.0018 |
| VS | 0.0357 ± 0.0006 | 0.0921 ± 0.0002 | 0.0938 ± 0.0107 | 0.1320 ± 0.0017 | 0.1352 ± 0.0021 | |
| DiCTS (Ours) | CRPS | 0.0829 ± 0.0006 | 0.2487 ± 0.0004 | 0.1579 ± 0.0023 | 0.2565 ± 0.0010 | 0.1934 ± 0.0010 |
| VS | 0.0321 ± 0.0003 | 0.0880 ± 0.0014 | 0.0988 ± 0.0026 | 0.1132 ± 0.0011 | 0.0959 ± 0.0038 |
- Noise-dependent routing has worse mean CRPS than standard self-attention on all five datasets, so slice-based aggregation alone does not explain the improvement of DiCTS.
- With the content path unchanged and only the routing replaced, history-guided routing improves mean CRPS on all five datasets and mean VS on four of five. On USHCN, noise-dependent routing has the lower mean VS (by 0.0050), which is within its seed standard deviation (0.0107).
- Against standard self-attention, the VS gains are not uniform: HiSSA improves VS on four of five datasets (not PhysioNet'12).
2. Counterfactual noise intervention. To test directly whether the grouping under noise-dependent routing is driven by the injected noise, we fix the observed history, query coordinates, ground-truth future, and diffusion noise level for each test instance, and independently resample the Gaussian noise eight times at each . We measure the assignment-induced mixing matrix where denotes the query-to-slice assignments and the slice masses, and report : the noise-induced change relative to the change under random query permutations (0 = noise-invariant; 1 = permutation reference).
Table. Noise-induced change in query grouping on five IMTS benchmarks (0 = noise-invariant; 1 = permutation reference).
| Dataset | Noise-dependent routing (range across ) | History-guided routing (by construction) |
|---|---|---|
| HumanActivity | 0.114–0.274 | 0.000 |
| PhysioNet'12 | 0.104–0.169 | 0.000 |
| USHCN | 0.505–0.679 | 0.000 |
| MIMIC-III | 0.133–0.252 | 0.000 |
| MIMIC-IV | 0.296–0.565 | 0.000 |
- Under noise-dependent routing, resampling the noise alone regroups queries on every dataset and at every noise level, by 10.4–67.9% of the permutation reference, although the history and the underlying future dynamics are unchanged.
- This confirms the premise of Section 3.3: under noise-dependent routing, which queries exchange information is determined in part by the injected noise rather than by the temporal evidence. Removing this dependence improves CRPS on all five datasets (1) and the band-wise energy score in 12 of 15 dataset–band cells (3).
Author Response 1: Summary
We thank the reviewer for the careful and constructive assessment. We provide additional experiments and clarifications addressing the following concerns:
We have also corrected the VS entries in Table 1 and revised the abstract, figure descriptions, notation, and limitations. Detailed results and discussions follow below.